-- Logs begin at Wed 2022-07-20 14:04:19 UTC, end at Wed 2022-07-20 15:39:03 UTC. -- Jul 20 14:38:41.025585 dv-cinder-849022-4-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 20 14:38:41.035639 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 14:38:41 2022] *** Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: nodename: dv-cinder-849022-4-iscsi Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: machine: x86_64 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: clock source: unix Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: pcre jit disabled Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: detected number of CPU cores: 4 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: current working directory: / Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: detected binary path: /usr/bin/uwsgi-core Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: your processes number limit is 47680 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: your memory page size is 4096 bytes Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: detected max file descriptor number: 2048 Jul 20 14:38:41.036615 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: lock engine: pthread robust mutexes Jul 20 14:38:41.037695 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: thunder lock: enabled Jul 20 14:38:41.037695 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 20 14:38:41.037695 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 14:38:41.080141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: Python main interpreter initialized at 0x56252c9319c0 Jul 20 14:38:41.080141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: python threads support enabled Jul 20 14:38:41.080141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: your server socket listen backlog is limited to 100 connections Jul 20 14:38:41.080141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: your mercy for graceful operations on workers is 90 seconds Jul 20 14:38:41.080484 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 14:38:41.080484 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: *** Operational MODE: preforking *** Jul 20 14:38:41.080786 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: *** uWSGI is running in multiple interpreter mode *** Jul 20 14:38:41.080786 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: spawned uWSGI master process (pid: 109855) Jul 20 14:38:41.081000 dv-cinder-849022-4-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 20 14:38:41.084141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: spawned uWSGI worker 1 (pid: 109856, cores: 1) Jul 20 14:38:41.084141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: spawned uWSGI worker 2 (pid: 109857, cores: 1) Jul 20 14:38:41.084141 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 14:38:41.724029 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 20 14:38:41.725149 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 20 14:38:42.819993 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:38:42.819993 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: from cryptography.utils import int_from_bytes Jul 20 14:38:42.820608 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:38:42.820608 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: from cryptography.utils import int_from_bytes Jul 20 14:38:42.822984 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:38:42.822984 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: from cryptography.utils import int_from_bytes Jul 20 14:38:42.823479 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:38:42.823479 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: from cryptography.utils import int_from_bytes Jul 20 14:38:45.240503 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:38:45.241098 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:38:45.241194 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:38:45.241529 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:38:45.241601 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:38:45.241676 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:38:45.241770 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:38:45.241840 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:38:45.242108 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:38:45.242252 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.242351 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:38:45.242482 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.242722 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.242870 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243021 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243134 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243387 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243521 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243662 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.243755 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244023 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244157 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244336 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244484 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244613 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244709 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.244894 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245041 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245209 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245352 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245486 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245584 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849022-4-iscsi {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245815 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.245959 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246126 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246258 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849022-4-iscsi {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246375 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246462 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246714 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.246847 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247001 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247139 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247329 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247437 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247621 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247719 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.247940 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.248104 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.248549 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.248684 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.248801 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.248928 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249035 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249148 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249295 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249409 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249607 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249737 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.249881 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250008 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250179 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250318 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250463 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250624 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250818 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.250954 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251124 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251249 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251423 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251562 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849022-4-iscsi {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251718 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.251848 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252114 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849022-4-iscsi {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252331 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252457 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252570 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252748 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.252879 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253058 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253208 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253445 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253573 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253756 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.253895 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.254026 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.254150 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.254356 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.254486 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255017 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255184 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255184 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255416 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255416 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255653 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255788 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.255913 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256054 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256179 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256302 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256424 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256560 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256684 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256806 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.256936 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257061 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257183 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257315 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257447 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257571 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257693 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257814 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.257953 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258099 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258225 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258400 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258532 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258715 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258863 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.258995 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259142 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259277 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259404 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259538 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259686 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.259892 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260022 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260155 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260278 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260417 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260541 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260664 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.260796 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261013 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261150 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261274 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261408 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261533 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261666 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261789 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.261922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262078 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262217 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262402 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262557 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262681 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.262814 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263011 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263174 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263309 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263433 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263634 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263778 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.263956 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264090 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.15 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264244 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.15 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264373 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264553 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264698 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.15 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.264875 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.15 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265004 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265178 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265316 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265449 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265573 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265771 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.265912 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266075 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266205 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266369 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266507 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266696 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266828 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.266951 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267129 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267275 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267399 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267585 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267717 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.267906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268034 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268167 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268413 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268551 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268676 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268800 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.268937 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269062 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269233 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269359 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269488 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269619 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269758 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.269906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270096 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270223 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270346 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270477 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270614 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270738 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.270875 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271014 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271266 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271396 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271532 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271657 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271779 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.271903 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272026 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272181 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272321 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272475 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272605 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272780 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.272912 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273074 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273205 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273395 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273531 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273686 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273816 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.273979 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274110 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274286 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274426 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274551 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274672 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.274870 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275009 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275167 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275290 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275476 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275615 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275785 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.275996 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276123 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276255 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276379 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276501 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276633 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276758 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.276946 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277082 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277217 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277347 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277536 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277674 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277797 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.277920 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278116 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278259 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278515 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278699 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278831 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.278984 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279165 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279307 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279431 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279603 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279734 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.279865 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280001 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280207 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280341 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280473 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280597 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280728 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.280861 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281018 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281148 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281357 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281482 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281608 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281741 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.281878 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282041 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282159 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282272 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282471 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282611 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282783 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.282913 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.283126 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.283256 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.283431 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:38:45.283563 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.283773 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.283912 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284230 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284430 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284563 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284746 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.284874 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285055 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285252 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285401 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285527 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285649 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285780 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.285967 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286114 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286241 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286380 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286555 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286686 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286860 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.286999 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287171 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287308 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287494 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287625 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287808 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.287969 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288225 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288408 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288546 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288733 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.288878 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289003 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289136 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289329 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289460 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289635 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289795 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.289960 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.290084 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.290208 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.290410 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.290536 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291016 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291243 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291399 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291587 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291721 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.291857 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292003 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292136 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292259 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292441 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292572 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292734 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.292867 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293048 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293177 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293362 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293682 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293812 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.293987 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294117 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294300 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294430 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294598 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294729 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.294908 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295039 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295218 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295340 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295545 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295676 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295799 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.295922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296124 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296262 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296427 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296558 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296740 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.296868 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297044 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297184 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297348 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297480 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297659 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297803 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.297930 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298142 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298277 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298402 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298528 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298658 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298791 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.298930 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299087 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299215 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299408 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299568 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299782 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.299920 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300055 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300181 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300358 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300565 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300701 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300828 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.300990 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301121 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301296 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301433 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301582 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301777 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.301904 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302027 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302150 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302274 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302464 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302595 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302748 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.302879 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303040 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303221 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303348 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303478 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303679 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303810 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.303981 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304123 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304247 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304369 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304570 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304700 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.304878 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305020 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305145 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305269 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305460 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305591 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305745 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.305884 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306060 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306188 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306372 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306510 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306644 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306767 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.306961 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307114 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307255 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307379 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307581 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307713 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.307901 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308030 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308208 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308338 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308477 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308602 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308817 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.308946 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309265 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309431 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309563 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309737 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.309867 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310044 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310188 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310313 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310437 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310649 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310783 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.310913 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311085 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311213 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311336 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311508 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311640 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311779 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.311985 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312112 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312235 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312358 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312482 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312797 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.312922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313044 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313236 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313368 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313500 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313624 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313817 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.313957 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314081 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314205 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314405 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314535 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314713 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.314843 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315036 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315237 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315426 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315555 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315734 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.315862 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316046 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316176 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316338 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316469 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316660 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316798 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.316954 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317098 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317273 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317403 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317534 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317854 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.317991 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318149 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318276 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318449 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318583 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318759 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.318887 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319093 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319222 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319387 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319515 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319687 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319822 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.319996 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320125 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320294 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320426 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320602 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320752 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.320941 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321075 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321201 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321324 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321516 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321654 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321830 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.321986 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322173 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322308 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322441 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322566 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322710 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.322844 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323009 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323157 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323333 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323469 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323636 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323761 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.323941 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324078 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324239 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324367 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324547 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324674 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324844 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.324979 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325153 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325279 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325450 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325584 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325758 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.325982 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326147 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326270 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326394 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326656 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326787 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.326920 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327101 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327256 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327384 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327557 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327697 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327870 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.327998 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328295 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328471 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328608 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328775 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.328906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329069 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329194 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329365 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329501 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329662 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329787 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.329962 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330088 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330259 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330384 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330583 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330710 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.330984 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331142 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331297 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331447 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331571 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331695 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331816 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.331945 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332109 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332323 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332451 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332574 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332700 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332837 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.332962 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333093 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333274 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333402 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333580 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333716 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.333874 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334001 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334170 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334297 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334482 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334624 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334778 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.334906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335089 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335219 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335389 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335531 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335685 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335819 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.335982 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336109 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336278 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336414 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336589 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336715 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.336896 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337033 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337194 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337320 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337494 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337628 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337802 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.337932 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338104 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338242 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338402 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338536 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338706 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.338846 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339008 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339144 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339334 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339461 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339638 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339779 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.339937 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340072 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340234 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340360 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340535 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340675 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340839 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.340968 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341167 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341313 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341438 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341572 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341697 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341824 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.341993 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342122 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342368 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342514 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342639 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342762 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.342892 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.343174 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.343320 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.343500 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.343757 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.343885 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344085 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344213 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344396 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344533 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344702 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.344828 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345016 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345142 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345329 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345457 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345656 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345791 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.345954 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346090 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346259 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346420 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346582 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346721 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.346882 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347010 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347209 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347336 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347508 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347649 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347806 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.347940 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348103 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348231 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348402 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348530 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348700 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348829 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.348996 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349131 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349326 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349452 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349576 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349698 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349858 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.349995 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350151 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350277 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350455 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350592 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350751 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.350886 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351098 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351306 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351433 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351557 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351680 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351819 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.351945 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352076 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352224 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352353 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352551 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352693 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352823 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.352947 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353142 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353271 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353445 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353574 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353777 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.353906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354074 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354209 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354373 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354502 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354694 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354822 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.354993 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355148 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355315 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355444 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355609 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355739 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.355910 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356036 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356205 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356332 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356498 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356625 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356798 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.356934 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357096 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357233 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357397 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357523 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357698 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357833 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.357988 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358127 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358287 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358415 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358583 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358719 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.358889 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359018 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359195 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359310 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359490 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359627 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359782 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.359909 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360079 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360206 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360375 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360501 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360671 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360799 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.360969 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361095 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361260 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361460 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361596 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361718 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.361874 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362058 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362222 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362358 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362530 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362721 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362848 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.362971 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363132 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363254 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363414 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363549 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363727 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.363875 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364010 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364136 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364311 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364439 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364606 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364742 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.364908 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365035 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365203 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365331 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365497 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365636 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365794 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.365928 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366093 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366220 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366397 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366533 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366689 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366844 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.366991 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367171 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367309 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367432 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367603 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367736 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.367898 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368024 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368194 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368327 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368490 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368616 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368786 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.368921 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369080 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369214 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369372 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369498 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369668 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369793 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.369965 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370100 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370261 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370394 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370555 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370691 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370853 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.370988 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371171 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371329 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371482 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371684 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371811 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.371934 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372057 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372189 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372357 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372494 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372766 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.372899 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373023 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373145 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373278 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373413 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373576 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373705 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.373878 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374004 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374171 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374298 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374476 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374603 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374773 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.374891 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375098 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375242 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375400 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375527 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375693 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375821 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.375988 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376115 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376286 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376423 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376584 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376719 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.376923 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377047 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377171 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377294 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377480 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377641 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377800 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.377934 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378104 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378232 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378406 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378542 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378706 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.378834 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379003 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379172 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379327 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379472 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379630 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379757 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.379932 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380059 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380234 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380370 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380536 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380662 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380833 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.380967 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381132 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381269 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381432 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381559 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381763 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.381894 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382029 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382161 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382320 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382447 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382617 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382751 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.382959 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383109 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383247 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383372 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383497 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383629 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383793 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.383922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384090 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384230 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384394 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384531 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384694 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.384822 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385081 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385211 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385379 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385517 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385680 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385815 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.385978 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386105 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386282 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386417 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386584 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386710 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.386876 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387009 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387224 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387369 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387523 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387650 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.387877 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388002 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388179 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388323 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388478 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388605 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388771 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.388898 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389071 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389207 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389366 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389492 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389665 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389790 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.389958 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390102 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390255 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390383 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390548 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390674 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390845 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.390972 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391206 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391334 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391459 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391581 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391764 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.391967 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392132 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392265 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392463 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392602 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392765 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.392892 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393100 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393250 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393460 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393598 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393773 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.393901 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394087 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394214 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394384 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394521 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394688 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.394905 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395041 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395186 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395310 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395445 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395614 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395752 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.395913 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396040 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396215 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396342 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396510 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396649 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396823 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.396950 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397123 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397251 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397423 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397550 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397737 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.397878 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398043 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398360 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398489 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398726 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.398852 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399183 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399375 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399504 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399672 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399815 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.399966 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400103 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400266 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400393 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400560 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400688 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.400858 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401002 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401161 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401287 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401455 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401581 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401755 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.401894 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402093 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402219 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402343 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402467 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402621 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402763 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.402917 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403070 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403269 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403396 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403520 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403657 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403832 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.403958 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404128 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404255 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404440 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404575 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404750 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.404876 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405043 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405178 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405355 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405491 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405652 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405785 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.405944 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406070 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406241 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406377 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406535 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406662 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.406835 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407024 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407235 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407362 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407487 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407767 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.407910 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408080 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408210 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408401 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408535 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408719 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.408901 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409081 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409217 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409410 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409538 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409718 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.409847 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410027 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410154 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410320 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410446 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410616 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410752 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.410915 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411062 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411234 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411364 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411529 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411665 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411825 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.411953 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412166 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412306 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412442 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412566 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412691 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412823 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.412969 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413096 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413251 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413438 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413565 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413689 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413851 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.413988 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414147 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414317 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414442 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414574 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414726 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.414930 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.415121 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.415248 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.415372 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.415494 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.415976 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416120 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416308 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416446 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416610 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416738 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.416918 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417067 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417234 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417367 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417531 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417663 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417847 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.417975 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418148 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418274 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418453 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418578 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418764 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.418892 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419088 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419219 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419398 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419533 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419705 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.419833 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420012 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420145 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420311 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420436 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420618 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420744 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.420922 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421046 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421232 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421359 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421538 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421736 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.421864 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422055 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422349 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422494 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422650 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422776 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.422954 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423137 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423277 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423428 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423617 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423755 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.423880 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424003 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424148 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424288 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424470 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424609 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424776 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.424903 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425086 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425212 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425393 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425522 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425708 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.425833 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426022 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426161 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426329 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426455 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426638 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426765 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.426948 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.427101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.427322 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.427499 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.427691 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.427884 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428072 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428250 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428250 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428548 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428696 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.428822 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429081 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429382 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429508 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429683 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429819 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.429988 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430115 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430292 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430419 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430592 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430734 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.430891 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431020 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431223 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431353 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431529 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431658 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431843 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.431971 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432167 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432296 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432513 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432648 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432781 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.432906 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433067 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433372 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433501 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433749 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.433884 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434065 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434192 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434380 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434510 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434696 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.434824 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435008 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435179 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435395 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435549 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435709 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.435865 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436029 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436199 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436328 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436524 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436660 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436848 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.436976 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437160 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437297 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437465 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437594 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437778 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.437906 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438079 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438216 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438375 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438686 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438823 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.438972 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439130 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439297 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439521 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439659 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439863 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.439991 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.440251 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.440390 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.440570 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.440697 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.440878 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441007 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441188 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441401 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441542 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441741 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.441870 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442058 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442186 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442364 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442703 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442840 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.442965 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443127 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443281 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443409 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443579 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443707 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.443906 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444041 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444167 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444290 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444496 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444624 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444747 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.444872 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445016 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445153 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445332 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445468 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445629 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445757 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.445924 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446061 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446219 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446347 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446520 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446661 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446813 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.446942 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447119 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447258 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447418 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447547 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447712 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.447850 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448014 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448141 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448305 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448445 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448603 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448730 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.448896 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449022 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449195 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449327 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449485 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449611 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449783 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.449911 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450077 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450203 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450381 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450516 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450683 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450810 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.450986 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451132 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451306 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451433 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451607 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451732 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.451919 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452047 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452222 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452359 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452524 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452651 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452854 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.452987 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453119 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453243 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453410 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453653 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453781 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.453998 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454141 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454266 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454391 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454591 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454718 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.454912 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455041 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455309 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455476 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455605 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455833 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.455973 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456152 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456289 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456480 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456609 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456795 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.456922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457095 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457225 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457394 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457522 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457690 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457817 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.457989 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458118 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458287 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458415 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458589 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458725 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.458887 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459013 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459212 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459351 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459511 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459640 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459809 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.459945 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460107 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460249 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460411 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460531 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460714 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.460842 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461020 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461157 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461324 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461452 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461627 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461756 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.461933 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462061 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462231 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462358 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462529 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462656 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.462901 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463097 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463227 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463351 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463475 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463599 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463740 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.463869 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464109 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464254 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464380 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464513 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464665 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464792 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.464970 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465098 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465282 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465421 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465603 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465729 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.465904 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466033 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466202 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466339 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466514 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466642 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466816 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.466997 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467183 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467318 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467443 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467567 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467749 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.467876 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468078 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468214 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468398 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468526 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468707 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.468841 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469001 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469128 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469298 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469425 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469592 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469728 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.469889 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470017 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470197 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470324 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470497 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470634 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470789 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.470916 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471100 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471232 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471388 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471583 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471768 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.471897 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472066 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472361 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472574 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472702 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472831 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.472970 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473160 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473297 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473421 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473552 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473677 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473799 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.473933 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.474073 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.474673 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.474796 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.474904 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475011 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475150 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475279 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475389 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475496 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475602 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475708 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475814 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.475930 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476083 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476215 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476327 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476434 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476556 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476673 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476781 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.476988 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.477105 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.479444 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.479610 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.479740 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.479856 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.479966 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480075 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480190 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480309 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480417 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480525 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480525 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480724 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480829 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.480951 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481058 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481164 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481268 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481374 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481489 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481618 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481725 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481831 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.481947 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.482625 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.482756 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.482884 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483009 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483178 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483330 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483440 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483514 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483597 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483675 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483754 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483823 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483904 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.483980 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484051 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484126 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484203 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484281 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484351 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484420 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484497 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484573 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484642 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484718 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484787 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484861 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.484932 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485000 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485066 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485146 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485224 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485293 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485369 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485501 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485653 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485798 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.485961 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.486107 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.486284 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.486408 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.486591 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.486752 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487206 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487365 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487506 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487646 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487798 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.487941 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488079 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488228 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488376 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488525 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488672 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488820 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.488961 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489098 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489246 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489394 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489534 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489672 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489819 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.489959 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490104 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490252 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490391 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490570 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490716 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.490855 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.491024 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.491232 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.491373 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.491512 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=109856) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:38:45.491650 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.491788 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.492053 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.492360 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.492666 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.492965 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:38:45.493203 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=109857) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:38:45.551651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO dbcounter [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Registered counter for database nova_api Jul 20 14:38:45.553707 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO dbcounter [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Registered counter for database nova_api Jul 20 14:38:45.668092 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_db.sqlalchemy.engines [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109856) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.669925 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_db.sqlalchemy.engines [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.675260 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writer thread running {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.676583 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writer thread running {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.729603 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.context [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f84c0e38-8de4-44d7-ae90-44f10e366750(cell1) {{(pid=109856) load_cells /opt/stack/nova/nova/context.py:464}} Jul 20 14:38:45.731030 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.context [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f84c0e38-8de4-44d7-ae90-44f10e366750(cell1) {{(pid=109857) load_cells /opt/stack/nova/nova/context.py:464}} Jul 20 14:38:45.733522 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:38:45.734933 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_concurrency.lockutils [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109857) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:38:45.735114 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:38:45.735861 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:38:45.736520 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_concurrency.lockutils [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=109857) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:38:45.737352 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_concurrency.lockutils [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109857) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:38:45.737987 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:38:45.739470 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_concurrency.lockutils [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=109857) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:38:45.739981 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO dbcounter [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Registered counter for database nova_cell0 Jul 20 14:38:45.741504 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO dbcounter [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Registered counter for database nova_cell0 Jul 20 14:38:45.743822 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO dbcounter [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Registered counter for database nova_cell1 Jul 20 14:38:45.745194 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO dbcounter [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Registered counter for database nova_cell1 Jul 20 14:38:45.751885 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_db.sqlalchemy.engines [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109856) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.753723 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_db.sqlalchemy.engines [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109856) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.755340 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_db.sqlalchemy.engines [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.758935 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_db.sqlalchemy.engines [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.761780 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writer thread running {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.764002 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writer thread running {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.766585 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writer thread running {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.766724 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writer thread running {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.905163 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO dbcounter [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Registered counter for database nova_cell0 Jul 20 14:38:45.909096 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO dbcounter [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Registered counter for database nova_cell0 Jul 20 14:38:45.915095 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_db.sqlalchemy.engines [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109856) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.919108 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG oslo_db.sqlalchemy.engines [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=109857) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:38:45.919953 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writer thread running {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:45.923688 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writer thread running {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:38:46.250874 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: WARNING keystonemiddleware.auth_token [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:38:46.253750 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: WARNING keystonemiddleware.auth_token [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:38:46.258389 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: WARNING keystonemiddleware.auth_token [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:38:46.261104 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: WARNING keystonemiddleware.auth_token [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:38:46.598204 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: WARNING keystonemiddleware.auth_token [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:38:46.601138 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: WARNING keystonemiddleware.auth_token [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:38:46.604354 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: WARNING keystonemiddleware.auth_token [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:38:46.607153 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: WARNING keystonemiddleware.auth_token [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:38:46.609512 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56252c9319c0 pid: 109857 (default app) Jul 20 14:38:46.611625 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:38:46.612530 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56252c9319c0 pid: 109856 (default app) Jul 20 14:38:46.612909 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-ac2046a5-d49b-40e8-b540-579c1abc40ae None None] 10.222.0.15 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001882 Jul 20 14:38:46.613325 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 788 bytes} [Wed Jul 20 14:38:46 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:02.818274 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:02.819515 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001980 Jul 20 14:40:02.820156 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:02 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:02.831934 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_api:SELECT=3 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:02.834887 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_cell0:SELECT=3 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:02.835870 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_cell1:SELECT=3 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:02.836669 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:03.696120 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-7236ce01-16b9-4a91-b8ab-8cd23c44ec0b admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:03.727548 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-7236ce01-16b9-4a91-b8ab-8cd23c44ec0b admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.901217 Jul 20 14:40:03.728289 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 2/3] 10.222.0.15 () {60 vars in 1291 bytes} [Wed Jul 20 14:40:02 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:05.942461 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:05.943548 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-4ea670f8-ed70-4568-96a2-c2bf7a2fcfcd None None] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001503 Jul 20 14:40:05.943936 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 2/4] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:06.411835 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-a59f7f87-b456-40d3-827f-4799622be633 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "cirros256", "id": "c1", "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0, "ram": 256, "os-flavor-access:is_public": true}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:06.429991 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:40:06.429991 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: warnings.warn(msg) Jul 20 14:40:06.457920 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-a59f7f87-b456-40d3-827f-4799622be633 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.506649 Jul 20 14:40:06.458593 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 3/5] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:05 2022] POST /compute/v2.1/flavors => generated 407 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:06.679618 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-9e5fc908-0b0a-412f-a5cd-c7fc8ad13987 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:06.682067 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:40:06.682067 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:40:06.700837 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_api:SELECT=3 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.701756 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_cell0:SELECT=3 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.702444 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_cell1:SELECT=3 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.703318 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:06.749991 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-9e5fc908-0b0a-412f-a5cd-c7fc8ad13987 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.282829 Jul 20 14:40:06.750550 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 3/6] 10.222.0.15 () {64 vars in 1335 bytes} [Wed Jul 20 14:40:06 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:08.972743 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-a59f7f87-b456-40d3-827f-4799622be633 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:08.973770 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-a59f7f87-b456-40d3-827f-4799622be633 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001623 Jul 20 14:40:08.974350 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 4/7] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:09.629581 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-e666736c-4161-4072-9b52-e0d420e11ab7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512, "os-flavor-access:is_public": true, "disk": 5, "name": "ds512M", "rxtx_factor": 1.0, "vcpus": 1, "id": "d1", "swap": 0}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:09.652902 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:40:09.652902 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:40:09.674413 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-e666736c-4161-4072-9b52-e0d420e11ab7 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.693972 Jul 20 14:40:09.675031 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 4/8] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:08 2022] POST /compute/v2.1/flavors => generated 404 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:40:09.687005 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-1a56b3ff-3388-4ebd-b44a-c1dd7af337c9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:09.688745 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:40:09.688745 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: warnings.warn(msg) Jul 20 14:40:09.722920 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-1a56b3ff-3388-4ebd-b44a-c1dd7af337c9 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039545 Jul 20 14:40:09.723569 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 5/9] 10.222.0.15 () {64 vars in 1335 bytes} [Wed Jul 20 14:40:09 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:11.916336 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-e666736c-4161-4072-9b52-e0d420e11ab7 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:11.917383 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-e666736c-4161-4072-9b52-e0d420e11ab7 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Jul 20 14:40:11.918720 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:12.289681 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-639041c2-3bcb-475d-99a5-0d8fed59d9f0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "disk": 10, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds1G", "swap": 0, "rxtx_factor": 1.0, "ram": 1024, "vcpus": 1}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:12.308272 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-639041c2-3bcb-475d-99a5-0d8fed59d9f0 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.384297 Jul 20 14:40:12.308910 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 6/11] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:11 2022] POST /compute/v2.1/flavors => generated 404 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:12.321011 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-55948f76-fa87-450b-8169-7a37def7865c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:12.344028 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-55948f76-fa87-450b-8169-7a37def7865c admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026783 Jul 20 14:40:12.344586 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 6/12] 10.222.0.15 () {64 vars in 1335 bytes} [Wed Jul 20 14:40:12 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:40:14.560279 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-639041c2-3bcb-475d-99a5-0d8fed59d9f0 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:14.561278 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-639041c2-3bcb-475d-99a5-0d8fed59d9f0 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001456 Jul 20 14:40:14.561694 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 7/13] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:14.987429 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-2737267e-188f-4a86-b736-675a5aa8ebeb admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "name": "ds2G", "vcpus": 2, "rxtx_factor": 1.0, "disk": 10, "id": "d3", "swap": 0}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:15.006915 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-2737267e-188f-4a86-b736-675a5aa8ebeb admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.439099 Jul 20 14:40:15.007595 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 7/14] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:14 2022] POST /compute/v2.1/flavors => generated 404 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:15.019943 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-97a40428-47f3-4fea-bc72-5cfebbc89343 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:15.049998 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-97a40428-47f3-4fea-bc72-5cfebbc89343 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033584 Jul 20 14:40:15.050971 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 8/15] 10.222.0.15 () {64 vars in 1335 bytes} [Wed Jul 20 14:40:15 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:40:17.307025 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-2737267e-188f-4a86-b736-675a5aa8ebeb admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:17.308057 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-2737267e-188f-4a86-b736-675a5aa8ebeb admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Jul 20 14:40:17.308423 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 8/16] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:17.717231 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 4096, "name": "ds4G", "swap": 0, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 4, "disk": 20}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:17.739633 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.424959 Jul 20 14:40:17.740547 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 9/17] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:40:17.753055 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-161ab18d-f678-4170-abe2-b1e0c91517f9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:17.777528 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-161ab18d-f678-4170-abe2-b1e0c91517f9 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028357 Jul 20 14:40:17.785759 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 9/18] 10.222.0.15 () {64 vars in 1335 bytes} [Wed Jul 20 14:40:17 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:20.113264 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:20.114820 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002019 Jul 20 14:40:20.115470 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 10/19] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:20.555328 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-3498ff64-8949-459e-906b-c6a169cacd0e admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:20.594383 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-3498ff64-8949-459e-906b-c6a169cacd0e admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.471839 Jul 20 14:40:20.595247 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 10/20] 10.222.0.15 () {60 vars in 1291 bytes} [Wed Jul 20 14:40:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:40:22.865764 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:22.867148 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-c442ae6b-d7b3-4c66-9e52-b04c51f2042e admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002785 Jul 20 14:40:22.867733 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 11/21] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:22 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:23.329953 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-089e0543-c9ed-47f9-aae6-776a5b07c934 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "disk": 1, "rxtx_factor": 1.0, "name": "m1.tiny", "ram": 512, "swap": 0, "id": "1"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:23.353350 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-089e0543-c9ed-47f9-aae6-776a5b07c934 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.479718 Jul 20 14:40:23.353900 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 11/22] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:22 2022] POST /compute/v2.1/flavors => generated 402 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:23.367518 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-b323c68f-65a9-4bd6-8335-e4a33073f0de admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:23.396507 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-b323c68f-65a9-4bd6-8335-e4a33073f0de admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033899 Jul 20 14:40:23.397602 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 12/23] 10.222.0.15 () {64 vars in 1332 bytes} [Wed Jul 20 14:40:23 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:25.710246 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-089e0543-c9ed-47f9-aae6-776a5b07c934 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:25.711288 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-089e0543-c9ed-47f9-aae6-776a5b07c934 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001474 Jul 20 14:40:25.711776 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 12/24] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:26.140290 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-9dffc7af-94cf-40eb-9c84-e4955daf0d19 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "rxtx_factor": 1.0, "swap": 0, "id": "2", "os-flavor-access:is_public": true, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.small", "vcpus": 1}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:26.159002 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-9dffc7af-94cf-40eb-9c84-e4955daf0d19 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.440895 Jul 20 14:40:26.159537 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 13/25] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:26.174784 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-5283d68b-73e0-4d9a-a7c3-d3e8780c3b23 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:26.200393 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-5283d68b-73e0-4d9a-a7c3-d3e8780c3b23 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030508 Jul 20 14:40:26.200979 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 13/26] 10.222.0.15 () {64 vars in 1332 bytes} [Wed Jul 20 14:40:26 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:28.403260 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-9dffc7af-94cf-40eb-9c84-e4955daf0d19 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:28.406161 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-9dffc7af-94cf-40eb-9c84-e4955daf0d19 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001589 Jul 20 14:40:28.406595 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 14/27] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:28.835110 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-3571be8d-1310-4506-b982-4fa2a9e1858e admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "disk": 40, "swap": 0, "name": "m1.medium", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 4096, "vcpus": 2, "rxtx_factor": 1.0}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:28.853025 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-3571be8d-1310-4506-b982-4fa2a9e1858e admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.441731 Jul 20 14:40:28.853605 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 14/28] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:28 2022] POST /compute/v2.1/flavors => generated 406 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:40:28.866070 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6cc27c0d-4182-4874-b69f-7051463c0e5a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:28.888407 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6cc27c0d-4182-4874-b69f-7051463c0e5a admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026256 Jul 20 14:40:28.888978 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 15/29] 10.222.0.15 () {64 vars in 1332 bytes} [Wed Jul 20 14:40:28 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:31.117010 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-3571be8d-1310-4506-b982-4fa2a9e1858e admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:31.118089 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-3571be8d-1310-4506-b982-4fa2a9e1858e admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001477 Jul 20 14:40:31.118534 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 15/30] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:31.568379 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d157710f-171c-4b51-ad5d-15f7d9e33385 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "m1.large", "vcpus": 4, "ram": 8192, "disk": 80, "rxtx_factor": 1.0, "id": "4", "os-flavor-access:is_public": true}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:31.589098 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d157710f-171c-4b51-ad5d-15f7d9e33385 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.464525 Jul 20 14:40:31.589919 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 16/31] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:31 2022] POST /compute/v2.1/flavors => generated 405 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:40:31.602123 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8f700b2d-48e3-4d4c-94f2-49c74c7492bc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:31.628284 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8f700b2d-48e3-4d4c-94f2-49c74c7492bc admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.027442 Jul 20 14:40:31.628284 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 16/32] 10.222.0.15 () {64 vars in 1332 bytes} [Wed Jul 20 14:40:31 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:40:33.853099 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d157710f-171c-4b51-ad5d-15f7d9e33385 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:33.854113 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d157710f-171c-4b51-ad5d-15f7d9e33385 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 Jul 20 14:40:33.854503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 17/33] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:34.306260 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "name": "m1.xlarge", "ram": 16384, "swap": 0, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "5"}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:34.330392 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.469405 Jul 20 14:40:34.331890 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 17/34] 10.222.0.15 () {64 vars in 1282 bytes} [Wed Jul 20 14:40:33 2022] POST /compute/v2.1/flavors => generated 408 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:40:34.347623 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-853b95fb-00ac-471d-9ed7-2140d74d121a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:40:34.351584 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=9,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:34.377357 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-853b95fb-00ac-471d-9ed7-2140d74d121a admin admin] 10.222.0.15 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037498 Jul 20 14:40:34.378329 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 18/35] 10.222.0.15 () {64 vars in 1332 bytes} [Wed Jul 20 14:40:34 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:40:54.652980 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:54.654016 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Jul 20 14:40:54.654409 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 18/36] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:40:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:54.668609 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_api:SELECT=5,nova_api:SAVEPOINT=1,nova_api:INSERT=1,nova_api:RELEASE=1 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:55.088291 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-0fa2217f-ef4d-4e27-96d9-5683654147c8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:55.107976 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-0fa2217f-ef4d-4e27-96d9-5683654147c8 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.446886 Jul 20 14:40:55.108640 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 19/37] 10.222.0.15 () {60 vars in 1291 bytes} [Wed Jul 20 14:40:54 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:40:56.219501 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:56.220509 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8fc4ddb5-e028-4a95-8658-8bd2eb3082c1 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Jul 20 14:40:56.220901 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 19/38] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:40:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:56.782861 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-2ed68e19-a826-44c2-8b59-9b277f356ba0 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:56.789458 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.wsgi [None req-2ed68e19-a826-44c2-8b59-9b277f356ba0 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-2ed68e19-a826-44c2-8b59-9b277f356ba0 admin admin] Request method failure captured: Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept: application/json Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept-Encoding: gzip, deflate Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Connection: keep-alive Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Content-Length: 0 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Host: 10.222.0.15 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-Api-Version: compute 2.90 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-System-Scope: None Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: User-Agent: python-novaclient Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Auth-Token: gAAAAABi2BP4tvyFyx9z6u9uvx4rMHpMBEz-bBl_0UopQepmgi3yEodPs5bmyzkEc2FhiVFCEl-s44RUD3I16Uq5iEkTQwzWubgX_AFPU-0ePR13oj1DQCLzu1V4Am4-luC9CVtZqYIjTcoJ5_9OMWBQ4AqTwpoWgMMDtb2mJUY34Ox6-8l6uwE Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Id: None Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Name: None Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Identity-Status: Confirmed Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Is-Admin-Project: True Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Id: default Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Name: Default Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Name: admin Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Role: admin,reader,member Jul 20 14:40:56.792882 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Roles: admin,reader,member Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant: admin Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Name: admin Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User: admin Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Id: default Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Name: Default Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Name: admin Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: method: > Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: exception: Flavor 42 could not be found. Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: action_args: {'id': '42'} Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:40:56.794503 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:56.795505 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.796704 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-2ed68e19-a826-44c2-8b59-9b277f356ba0 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=109857) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:40:56.797793 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-2ed68e19-a826-44c2-8b59-9b277f356ba0 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.436255 Jul 20 14:40:56.797793 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 20/39] 10.222.0.15 () {60 vars in 1190 bytes} [Wed Jul 20 14:40:56 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 438 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:56.806728 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-35d1e078-49aa-4611-b2a1-48632c70f1bf admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:56.811164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:40:56.832898 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-35d1e078-49aa-4611-b2a1-48632c70f1bf admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.031628 Jul 20 14:40:56.833439 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 20/40] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:40:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:40:56.843981 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d8cad16f-6d7b-4293-9bcf-8ae56fe07c95 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:56.866586 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d8cad16f-6d7b-4293-9bcf-8ae56fe07c95 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026404 Jul 20 14:40:56.867188 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 21/41] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:40:56 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:40:56.879710 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-d0ed083e-d22a-45f5-9115-aaa2366d1ec8 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:56.886723 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.wsgi [None req-d0ed083e-d22a-45f5-9115-aaa2366d1ec8 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-d0ed083e-d22a-45f5-9115-aaa2366d1ec8 admin admin] Request method failure captured: Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept: application/json Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept-Encoding: gzip, deflate Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Connection: keep-alive Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Content-Length: 0 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Host: 10.222.0.15 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-Api-Version: compute 2.90 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-System-Scope: None Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: User-Agent: python-novaclient Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Auth-Token: gAAAAABi2BP4tvyFyx9z6u9uvx4rMHpMBEz-bBl_0UopQepmgi3yEodPs5bmyzkEc2FhiVFCEl-s44RUD3I16Uq5iEkTQwzWubgX_AFPU-0ePR13oj1DQCLzu1V4Am4-luC9CVtZqYIjTcoJ5_9OMWBQ4AqTwpoWgMMDtb2mJUY34Ox6-8l6uwE Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Id: None Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Name: None Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Identity-Status: Confirmed Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Is-Admin-Project: True Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Id: default Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Name: Default Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Name: admin Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Role: admin,reader,member Jul 20 14:40:56.890102 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Roles: admin,reader,member Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant: admin Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Name: admin Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User: admin Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Id: default Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Name: Default Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Name: admin Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: method: > Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: exception: Flavor 42 could not be found. Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: action_args: {'id': '42'} Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:40:56.891656 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:56.892517 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:56.894135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-d0ed083e-d22a-45f5-9115-aaa2366d1ec8 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=109856) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:40:56.895424 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-d0ed083e-d22a-45f5-9115-aaa2366d1ec8 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.016084 Jul 20 14:40:56.895424 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 21/42] 10.222.0.15 () {60 vars in 1190 bytes} [Wed Jul 20 14:40:56 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:57.873495 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d8cad16f-6d7b-4293-9bcf-8ae56fe07c95 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:57.874529 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d8cad16f-6d7b-4293-9bcf-8ae56fe07c95 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Jul 20 14:40:57.874889 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 22/43] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:40:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:40:58.423959 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8063b8b8-0cd1-4798-9f3c-8e1c9cfa91d8 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:58.430819 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.wsgi [None req-8063b8b8-0cd1-4798-9f3c-8e1c9cfa91d8 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8063b8b8-0cd1-4798-9f3c-8e1c9cfa91d8 admin admin] Request method failure captured: Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept: application/json Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept-Encoding: gzip, deflate Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Connection: keep-alive Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Content-Length: 0 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Host: 10.222.0.15 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-Api-Version: compute 2.90 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-System-Scope: None Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: User-Agent: python-novaclient Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Auth-Token: gAAAAABi2BP5eaIfS1754MPz-kxC_woI2g5arEe5xiw1kZ6j7pu8pZnakTwZMjjur7JcfO2jCJsNYyj1GhtaOeyJoCq3AY2V9XNndzeaivmAIAUz2wNr_P50J18l7b_UamEHltbnxE6VvXAo5HPShzr5Lmsl1fmK7-M-2xB3yGCRfJtfmj8jzCU Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Id: None Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Name: None Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Identity-Status: Confirmed Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Is-Admin-Project: True Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Id: default Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Name: Default Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Name: admin Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Role: admin,reader,member Jul 20 14:40:58.432620 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Roles: admin,reader,member Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant: admin Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Name: admin Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User: admin Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Id: default Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Name: Default Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Name: admin Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: method: > Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: exception: Flavor 84 could not be found. Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: action_args: {'id': '84'} Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:40:58.434158 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:58.435172 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.436487 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8063b8b8-0cd1-4798-9f3c-8e1c9cfa91d8 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=109856) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:40:58.437824 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8063b8b8-0cd1-4798-9f3c-8e1c9cfa91d8 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.424051 Jul 20 14:40:58.437824 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1190 bytes} [Wed Jul 20 14:40:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 426 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:58.444406 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-17e10a48-f0f7-4dd4-84af-39d16eb1fd26 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:58.460753 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-17e10a48-f0f7-4dd4-84af-39d16eb1fd26 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020102 Jul 20 14:40:58.461444 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 23/45] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:40:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:40:58.472362 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-816e366d-a3e1-46f1-98ed-0645eddf76b4 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:58.486297 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-816e366d-a3e1-46f1-98ed-0645eddf76b4 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017457 Jul 20 14:40:58.486839 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:40:58 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:40:58.496627 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-e13b9331-e5ca-4825-87b3-3c4977499bb3 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:58.502515 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.wsgi [None req-e13b9331-e5ca-4825-87b3-3c4977499bb3 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-e13b9331-e5ca-4825-87b3-3c4977499bb3 admin admin] Request method failure captured: Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept: application/json Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept-Encoding: gzip, deflate Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Connection: keep-alive Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Content-Length: 0 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Host: 10.222.0.15 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-Api-Version: compute 2.90 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-System-Scope: None Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: User-Agent: python-novaclient Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Auth-Token: gAAAAABi2BP5eaIfS1754MPz-kxC_woI2g5arEe5xiw1kZ6j7pu8pZnakTwZMjjur7JcfO2jCJsNYyj1GhtaOeyJoCq3AY2V9XNndzeaivmAIAUz2wNr_P50J18l7b_UamEHltbnxE6VvXAo5HPShzr5Lmsl1fmK7-M-2xB3yGCRfJtfmj8jzCU Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Id: None Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Name: None Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Identity-Status: Confirmed Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Is-Admin-Project: True Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Id: default Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Name: Default Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Name: admin Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Role: admin,reader,member Jul 20 14:40:58.504087 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Roles: admin,reader,member Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant: admin Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Name: admin Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User: admin Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Id: default Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Name: Default Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Name: admin Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: method: > Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: exception: Flavor 84 could not be found. Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: action_args: {'id': '84'} Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:40:58.505500 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:58.506392 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:40:58.507977 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-e13b9331-e5ca-4825-87b3-3c4977499bb3 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=109857) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:40:58.509298 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-e13b9331-e5ca-4825-87b3-3c4977499bb3 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012464 Jul 20 14:40:58.509298 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 24/47] 10.222.0.15 () {60 vars in 1190 bytes} [Wed Jul 20 14:40:58 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:40:59.505593 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-816e366d-a3e1-46f1-98ed-0645eddf76b4 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:40:59.506599 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-816e366d-a3e1-46f1-98ed-0645eddf76b4 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001449 Jul 20 14:40:59.506992 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 24/48] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:40:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:41:00.051931 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-a6374552-6458-4c9e-8829-e1297dbfa7e7 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:00.058007 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.wsgi [None req-a6374552-6458-4c9e-8829-e1297dbfa7e7 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-a6374552-6458-4c9e-8829-e1297dbfa7e7 admin admin] Request method failure captured: Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept: application/json Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Accept-Encoding: gzip, deflate Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Connection: keep-alive Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Content-Length: 0 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Host: 10.222.0.15 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-Api-Version: compute 2.90 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Openstack-System-Scope: None Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: User-Agent: python-novaclient Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Auth-Token: gAAAAABi2BP7GDe6JzBiNpCw118RzwGISjsTxpEICZe-zmhmjhs7qPlOoVVCM7eSkX86t6lLqa0AkarUL4ZjrWVXhox08cE9btn0aOhk_XZsJO0ZpbVPU9oW1iZZkSpMBdCsZxg234fc3mvz47pjrXLtW1vem94hjktwsg8_eWS0BrRT6KPFMtE Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Id: None Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Domain-Name: None Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Identity-Status: Confirmed Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Is-Admin-Project: True Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Id: default Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Domain-Name: Default Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Project-Name: admin Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Role: admin,reader,member Jul 20 14:41:00.059525 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Roles: admin,reader,member Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant: admin Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-Tenant-Name: admin Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User: admin Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Id: default Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Domain-Name: Default Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: X-User-Name: admin Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: method: > Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: exception: Flavor 451 could not be found. Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: action_args: {'id': '451'} Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:41:00.061101 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:41:00.062196 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.063621 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-a6374552-6458-4c9e-8829-e1297dbfa7e7 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=109857) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:41:00.064872 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-a6374552-6458-4c9e-8829-e1297dbfa7e7 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.419517 Jul 20 14:41:00.064872 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 25/49] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:40:59 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 421 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:00.070547 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-02997f13-db09-4b26-95ef-a0adebd9140f admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:00.084399 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-02997f13-db09-4b26-95ef-a0adebd9140f admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017408 Jul 20 14:41:00.084910 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 25/50] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:41:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:41:00.094113 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:00.109178 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018590 Jul 20 14:41:00.109688 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 26/51] 10.222.0.15 () {60 vars in 1210 bytes} [Wed Jul 20 14:41:00 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:41:00.119252 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8d6aa3bf-beef-4c77-9168-11d17822ee0a admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:00.125341 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.wsgi [None req-8d6aa3bf-beef-4c77-9168-11d17822ee0a admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8d6aa3bf-beef-4c77-9168-11d17822ee0a admin admin] Request method failure captured: Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept: application/json Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Accept-Encoding: gzip, deflate Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Connection: keep-alive Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Content-Length: 0 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Host: 10.222.0.15 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-Api-Version: compute 2.90 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Openstack-System-Scope: None Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: User-Agent: python-novaclient Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Auth-Token: gAAAAABi2BP7GDe6JzBiNpCw118RzwGISjsTxpEICZe-zmhmjhs7qPlOoVVCM7eSkX86t6lLqa0AkarUL4ZjrWVXhox08cE9btn0aOhk_XZsJO0ZpbVPU9oW1iZZkSpMBdCsZxg234fc3mvz47pjrXLtW1vem94hjktwsg8_eWS0BrRT6KPFMtE Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Id: None Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Domain-Name: None Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Identity-Status: Confirmed Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Is-Admin-Project: True Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Openstack-Nova-Api-Version: 2.90 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Id: default Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Domain-Name: Default Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Project-Name: admin Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Role: admin,reader,member Jul 20 14:41:00.127651 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Roles: admin,reader,member Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3ca83facd1c8420dacb9e91e8abe0fd4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3ca83facd1c8420dacb9e91e8abe0fd4"}]}] Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant: admin Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Id: 3ca83facd1c8420dacb9e91e8abe0fd4 Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-Tenant-Name: admin Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User: admin Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Id: default Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Domain-Name: Default Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Id: 651b2280231048d59d156e569aa33194 Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: X-User-Name: admin Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: method: > Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: exception: Flavor 451 could not be found. Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: action_args: {'id': '451'} Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:41:00.129164 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:41:00.130038 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: ERROR nova.api.openstack.wsgi Jul 20 14:41:00.131177 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-8d6aa3bf-beef-4c77-9168-11d17822ee0a admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=109856) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:41:00.132203 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-8d6aa3bf-beef-4c77-9168-11d17822ee0a admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.013345 Jul 20 14:41:00.132203 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 26/52] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:41:00 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 20 14:41:01.110712 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:01.111735 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Jul 20 14:41:01.112121 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 27/53] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:41:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:41:01.692356 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-71bc2fc4-95e3-43dc-9955-77dcce5c0b20 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:41:01.713223 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-71bc2fc4-95e3-43dc-9955-77dcce5c0b20 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.462997 Jul 20 14:41:01.713772 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 27/54] 10.222.0.15 () {64 vars in 1235 bytes} [Wed Jul 20 14:41:01 2022] POST /compute/v2.1/flavors => generated 404 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:41:02.702289 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:02.703305 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001426 Jul 20 14:41:02.703685 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 28/55] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:41:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:41:03.266223 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-2b025c1b-ea62-48e0-97b5-1ace82335646 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:41:03.285135 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-2b025c1b-ea62-48e0-97b5-1ace82335646 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.437321 Jul 20 14:41:03.285730 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 28/56] 10.222.0.15 () {64 vars in 1235 bytes} [Wed Jul 20 14:41:02 2022] POST /compute/v2.1/flavors => generated 405 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:41:04.278851 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:04.279866 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 Jul 20 14:41:04.280249 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 29/57] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:41:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:41:04.850051 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-5228682e-49dc-4740-980f-0f0a67e6377f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:41:04.869663 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-5228682e-49dc-4740-980f-0f0a67e6377f admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.452884 Jul 20 14:41:04.870191 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 29/58] 10.222.0.15 () {64 vars in 1235 bytes} [Wed Jul 20 14:41:04 2022] POST /compute/v2.1/flavors => generated 407 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:41:05.854970 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:05.856037 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001495 Jul 20 14:41:05.856418 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 30/59] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:41:05 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:41:06.440888 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-4ea27b3f-18c3-4e4e-b589-50d6b5e339b5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:41:06.467893 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-4ea27b3f-18c3-4e4e-b589-50d6b5e339b5 admin admin] 10.222.0.15 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.466892 Jul 20 14:41:06.468521 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 30/60] 10.222.0.15 () {64 vars in 1235 bytes} [Wed Jul 20 14:41:05 2022] POST /compute/v2.1/flavors => generated 409 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:41:07.440579 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:07.441682 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 Jul 20 14:41:07.442064 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 31/61] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:41:07 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:41:08.005454 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-dc30494a-eb68-44b8-91af-1784360146ae admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:41:08.024440 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-dc30494a-eb68-44b8-91af-1784360146ae admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.446220 Jul 20 14:41:08.024968 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 31/62] 10.222.0.15 () {60 vars in 1202 bytes} [Wed Jul 20 14:41:07 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:42:00.417290 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.419120 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-d6df7f86-c5a0-431a-ab27-21f14c1a1d02 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002611 Jul 20 14:42:00.419778 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 32/63] 10.222.0.15 () {54 vars in 930 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:42:00.434011 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:00.836350 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-e799e851-d432-401b-84d7-56f0afd04c8e admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.855464 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-e799e851-d432-401b-84d7-56f0afd04c8e admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.429082 Jul 20 14:42:00.856022 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 32/64] 10.222.0.15 () {60 vars in 1291 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:42:00.878171 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG dbcounter [-] [109857] Writing DB stats nova_api:SELECT=14 {{(pid=109857) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:42:00.883835 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-878e690a-efb4-41be-838e-ad6ef00d4d82 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.895939 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-878e690a-efb4-41be-838e-ad6ef00d4d82 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.019989 Jul 20 14:42:00.896655 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 33/65] 10.222.0.15 () {60 vars in 1282 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:42:00.909589 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-1edf6c88-007f-4bd4-8e6c-224ee69d11a8 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.917897 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-1edf6c88-007f-4bd4-8e6c-224ee69d11a8 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011860 Jul 20 14:42:00.918453 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 33/66] 10.222.0.15 () {60 vars in 1285 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:42:00.929649 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.937964 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011799 Jul 20 14:42:00.938478 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 34/67] 10.222.0.15 () {60 vars in 1285 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:42:00.949720 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-5afb0567-4598-4e07-90a6-8ea592da64b2 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:42:00.958133 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-5afb0567-4598-4e07-90a6-8ea592da64b2 admin admin] 10.222.0.15 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011854 Jul 20 14:42:00.958611 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 34/68] 10.222.0.15 () {60 vars in 1282 bytes} [Wed Jul 20 14:42:00 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:45:41.007922 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:41.008889 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Jul 20 14:45:41.009296 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:45:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:45:41.150974 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_api:SELECT=6 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:41.340885 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:41.342096 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:45:41.342096 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:45:41.343631 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:41.344109 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:41.344918 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:41.345418 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:41.383520 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-9fb7a375-02ac-4e0c-ad5a-8ed0580af151 admin admin] 10.222.0.15 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.233976 Jul 20 14:45:41.384142 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 35/70] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:45:41 2022] GET /compute/v2.1/os-services => generated 791 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:45:42.230302 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:42.231440 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001581 Jul 20 14:45:42.231893 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 36/71] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:45:42 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:45:42.463689 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:42.465008 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:45:42.465008 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:45:42.466398 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:42.466832 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:42.467904 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:42.468508 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:42.494228 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-2122d4e4-f464-48d5-964a-acc3b86095af admin admin] 10.222.0.15 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.094947 Jul 20 14:45:42.494877 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 36/72] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:45:42 2022] GET /compute/v2.1/os-services => generated 791 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:45:53.726806 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:53.727871 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001532 Jul 20 14:45:53.728268 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 37/73] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:45:53 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:45:53.870747 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_cell0:SELECT=4 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:53.871513 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_cell1:SELECT=4 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:53.872084 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG dbcounter [-] [109856] Writing DB stats nova_api:SELECT=8 {{(pid=109856) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:45:53.925105 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:53.926350 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:45:53.926350 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:45:53.927574 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:53.928004 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:53.928768 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:53.929185 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:53.954551 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-0e46e1f1-5c1a-4d5e-b893-9e46608ce5e1 admin admin] 10.222.0.15 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.088192 Jul 20 14:45:53.955169 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 37/74] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:45:53 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:45:54.817204 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: DEBUG nova.api.openstack.wsgi [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] Calling method '>' {{(pid=109857) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:54.818945 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: INFO nova.api.openstack.requestlog [None req-6ce55271-c642-4083-8954-aa30e25d41b8 admin admin] 10.222.0.15 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002384 Jul 20 14:45:54.819646 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: [pid: 109857|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1074 bytes} [Wed Jul 20 14:45:54 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:45:55.023374 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG nova.api.openstack.wsgi [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] Calling method '>' {{(pid=109856) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:45:55.025224 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 14:45:55.025224 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: warnings.warn(msg) Jul 20 14:45:55.027482 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:55.028107 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:55.029105 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:45:55.029547 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: DEBUG oslo_concurrency.lockutils [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=109856) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:45:55.052825 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: INFO nova.api.openstack.requestlog [None req-2b91a6e7-fbaf-439c-9070-47a8d9cd880d admin admin] 10.222.0.15 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.092824 Jul 20 14:45:55.053444 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: [pid: 109856|app: 0|req: 38/76] 10.222.0.15 () {60 vars in 1193 bytes} [Wed Jul 20 14:45:54 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:46:56.388659 dv-cinder-849022-4-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 20 14:46:56.389210 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: Wed Jul 20 14:46:56 2022 - graceful shutdown triggered... Jul 20 14:46:56.389648 dv-cinder-849022-4-iscsi devstack@n-api.service[109857]: Gracefully killing worker 2 (pid: 109857)... Jul 20 14:46:56.390053 dv-cinder-849022-4-iscsi devstack@n-api.service[109856]: Gracefully killing worker 1 (pid: 109856)... Jul 20 14:46:57.389708 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: worker 1 buried after 1 seconds Jul 20 14:46:57.389708 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: worker 2 buried after 1 seconds Jul 20 14:46:57.389708 dv-cinder-849022-4-iscsi devstack@n-api.service[109855]: goodbye to uWSGI. Jul 20 14:46:57.391381 dv-cinder-849022-4-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 20 14:46:57.391997 dv-cinder-849022-4-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 20 14:46:57.394856 dv-cinder-849022-4-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 20 14:46:57.404553 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 20 14:46:57.407175 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 14:46:57 2022] *** Jul 20 14:46:57.407321 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 14:46:57.407455 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 14:46:57.407565 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: nodename: dv-cinder-849022-4-iscsi Jul 20 14:46:57.407670 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: machine: x86_64 Jul 20 14:46:57.407774 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: clock source: unix Jul 20 14:46:57.407863 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: pcre jit disabled Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: detected number of CPU cores: 4 Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: current working directory: / Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: detected binary path: /usr/bin/uwsgi-core Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: your processes number limit is 47680 Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: your memory page size is 4096 bytes Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: detected max file descriptor number: 2048 Jul 20 14:46:57.408031 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: lock engine: pthread robust mutexes Jul 20 14:46:57.408522 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: thunder lock: enabled Jul 20 14:46:57.408522 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 20 14:46:57.408522 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 14:46:57.452439 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: Python main interpreter initialized at 0x55a17a6399c0 Jul 20 14:46:57.452439 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: python threads support enabled Jul 20 14:46:57.452439 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: your server socket listen backlog is limited to 100 connections Jul 20 14:46:57.452439 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: your mercy for graceful operations on workers is 90 seconds Jul 20 14:46:57.452865 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 14:46:57.452865 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: *** Operational MODE: preforking *** Jul 20 14:46:57.453187 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: *** uWSGI is running in multiple interpreter mode *** Jul 20 14:46:57.453187 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: spawned uWSGI master process (pid: 124378) Jul 20 14:46:57.453409 dv-cinder-849022-4-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 20 14:46:57.455870 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: spawned uWSGI worker 1 (pid: 124396, cores: 1) Jul 20 14:46:57.455870 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: spawned uWSGI worker 2 (pid: 124397, cores: 1) Jul 20 14:46:57.455870 dv-cinder-849022-4-iscsi devstack@n-api.service[124378]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 14:46:58.130423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 20 14:46:58.131118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 20 14:46:59.316701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:46:59.316701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: from cryptography.utils import int_from_bytes Jul 20 14:46:59.317490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:46:59.317490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: from cryptography.utils import int_from_bytes Jul 20 14:46:59.323927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:46:59.323927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: from cryptography.utils import int_from_bytes Jul 20 14:46:59.324380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 14:46:59.324380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: from cryptography.utils import int_from_bytes Jul 20 14:47:01.632190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:47:01.632781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:47:01.632989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:47:01.633322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:47:01.633578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:47:01.633967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.634293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.634592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.634917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.635273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.635591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.635937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.636234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.636559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.636847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.637163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.637461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849022-4-iscsi {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.637784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.638118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.638401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.638721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.639026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.639377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.639680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.640196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.640663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.641178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.641369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.641574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.641932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.642244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.642553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.642878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.643225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.643541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.643941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849022-4-iscsi {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.644263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.644570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.644887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.645305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.645615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.645928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.646230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.646554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.646865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.647215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.647539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.647861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.648184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.648483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.648789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.649100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.649422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.649726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.650033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.650282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.650593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.650912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.651264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.651590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.651843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.652092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.652410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.652731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.653037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.653348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.653653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.653960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.654266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.654573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.654885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.655212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.655547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.655865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.656275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.656558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.656814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.657212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.15 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.657524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.15 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.657831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.658171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.658491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.659130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.659466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.659794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.660103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.660412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.660720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.661040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.661376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.666364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.667194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.667194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.667194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.667400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.668055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.668055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.668217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.668505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.668788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.669458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.669458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.669707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.669970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.670275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.670622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.670927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.671256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.671578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.671879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.672185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.672501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.672811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.673146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.673436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.673742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.674047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.674352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.674657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.674958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.675284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.675606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.675914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.676219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.676379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 14:47:01.676875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.677031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 14:47:01.677175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.677175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.677464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.677634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 14:47:01.677784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.677941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 14:47:01.678085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 14:47:01.678237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.678389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.678537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.678712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.678961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.679148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.679395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.679544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.679693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.679870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.680043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.680273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.680544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.680693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.680837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.681005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.681157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.681310 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.681455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.681597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.681597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.681907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.682050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.682199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.682349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-849022-4-iscsi {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.682503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.682659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.682802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.682950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.683130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.683280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.683430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.683583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.683865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.684036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.684192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.684336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.684485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.684647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.684790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.684972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.685143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.685295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.685452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.685596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.685756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.685941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.686088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.686231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.686381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.686536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.686680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.686830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.687022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.687228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.687382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.687534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.687678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.687846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.687989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.688180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.688180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.688473 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-849022-4-iscsi {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.688814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.688979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.689138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.689282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.689516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.689674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.689819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.689970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.690147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.690395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.690569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.690825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.690975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.691166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.691322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.691472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.691631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.691784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.691935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.692094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.692266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.692409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.692557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.692706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.692857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.693008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.693157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.693307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.693456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.693598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.693749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.693903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.694055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.694207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.694357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.694507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.694658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.694809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.694960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.695125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.695285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.695448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.695590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.695741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.695890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.696041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.696194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.696345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.696506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.696663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.696816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.696958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.697120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.697263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.697427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.697578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.697722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.697881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.698032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.698185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.698342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.698492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.698492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.698778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.698926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.699134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.699294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.699447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.699686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.699834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.699995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.700161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.700314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.700479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.700623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.700764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.701023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.701182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.701327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.701477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.701635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.701778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.701778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.702753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.15 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.702906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.703075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.15 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.703075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.703373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.703373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.703662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.703813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.703955 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.704105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.704255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.704255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.704559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.704559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.704838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.704838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.705106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.705262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.705421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.705573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.705714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.705856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.705998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.706139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.706369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.706531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.706691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.706841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.706993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.707177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.707333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.707476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.707628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.707777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.707920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.708075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.708218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.708367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.708518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.708669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.708819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.708975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.709126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.709269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.709410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.709559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.709729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.709956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.710110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.710264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.710415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.710577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.710728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.710885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.711088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.711321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.711469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.711611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.711767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.711920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.712072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.712214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.712364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.712525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.712687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.712838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.712982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.713132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.713282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.713432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.713590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.713733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.713885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.714041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.714192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.714344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.714503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.714663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.714814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.714971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.715164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.715318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.715462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.715612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.715754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.715910 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.716061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.716203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.716345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.716495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.716649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.716800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.716951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.717102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.717245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.717395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.717552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.717694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.717843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.717993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.718144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.718287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.718447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.718598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.718741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.718883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.719025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.719232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.719389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.719541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.719700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.719859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.720186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.720347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.720493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.720695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.720880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.721100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.721262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.721518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.721676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.721827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.721980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.722131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.722131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.722416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.722416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.722698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 14:47:01.722860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.723939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.724091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.724253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.724967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.724967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.725262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.725405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.725563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.725713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.725855 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.726917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.727869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.728912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.729971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.730113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.730352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.730508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.730700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.730852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.731945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732397 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.732993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.733931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.734853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.735940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.736989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.737895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.738923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.739094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.739094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.739382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.739524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.739524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.741947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.742892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.743998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.744987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.745996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.746902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.747981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.748862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.749907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.750140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.750301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.750463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.750608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.750775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.751994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.752905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.753945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.754859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.755977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.756891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.757984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.758965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.759876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.760929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.761951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.762994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.763119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.763373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.763476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.763710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.763802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.764270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.764427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.764592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.764748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.764748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.765045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.765204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.765354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.765658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.765904 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.766989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.767980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.768280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.768280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.768581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.768731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.768888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.769047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.769198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.769375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.769710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.769874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.770966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.771901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.772137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.772522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.772661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.772873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.773949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.774925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.775959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.776983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.777921 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.778727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.779505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.779658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.779797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.779934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780546 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.780957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.781909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.782040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.782176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.782305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.784463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.784614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.784614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.784870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.785962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.786895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.787875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.788938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.789866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.790920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.791893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.792957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.793898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.794925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.795872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.796929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.797845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.798891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.799915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800681 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.800888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.801925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.802955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.803979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.804902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.805067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.805315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.805480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.805612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.805740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.806849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.807983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.808111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.808277 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.808472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.808629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.808885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.809917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.810382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.810553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.810719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.810865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.811964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.812892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.813840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814154 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.814978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.815246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.815415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.815574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.815730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.815893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.816791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.817888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.818983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.819953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.820994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.821993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.823902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.824928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.825192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.825444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.825700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.825935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.826177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.826420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.826654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.826891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.827156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.827387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.828919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.829138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.829366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.829525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.829688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.829867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.830926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.831843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.832164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.832316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.832462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.832613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.832805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.833034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.833249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.833476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.833705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.833952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.834168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.834383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.834595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.834808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.835039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.835391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.835620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.835844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.836986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.837434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.837652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.837866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.838093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.838093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.838514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.838745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.838745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.839239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.839461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.839681 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.839929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.839929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.840323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.840541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.840747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.840972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.841180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.841395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.841395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.841806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.842024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.842024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.842457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.842678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.842908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.843166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.843384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.843591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.843802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.844011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.844238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.844454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.844680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.844945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.845273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.845498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.845723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.845929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.846146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.846361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.846573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.846573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.846998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.847239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.847449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.847668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.847873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.848099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.848099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.848531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.848800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.849010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.849225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.849225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.849225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.849225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.850029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.850234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.850452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.850668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.850899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.851172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.851383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.851588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.851804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.852042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.852258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.852477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.852690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.852894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.853116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.853323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.853538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.853759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.853962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.854165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.854382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.854597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.854811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.855016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.855258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.855478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.855696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.855901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.856116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.856329 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.856559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.856763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.856985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.857194 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.857408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.857613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.857819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.858035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.858239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.858444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.858712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.858933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.859224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.859459 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.859710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.859946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.860166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.860371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.860638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.860868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.861076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.861280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.861486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.861692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.861907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.862139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.862355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.862560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.862776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.862992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.863240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.863446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.863651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.863859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.864065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.864267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.864604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.864813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.865044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.865249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.865456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.865672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.865888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.866123 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.866339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.866556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.866767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.866981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.867231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.867446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.867650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.867866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.867866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.868288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.868288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.868684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.868889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.869093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.869311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.869532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.869743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.869960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.870178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.870390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.870608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.870825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.871029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.871302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.871507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.871711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.871917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.872133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.872340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.872555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.872760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.872974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.873207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.873424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.873628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.873843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.874049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.874267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.874499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.874706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.874921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.875165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.875375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.875590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.875813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.876048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.876286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.876498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.876716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.876941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.877147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.877362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.877572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.877787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.878012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.878238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.878452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.878680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.878898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.879141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.879353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.879576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.879794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.879999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.880203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.880203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.880203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.880798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.881013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.881218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.881436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.881436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.881853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.882060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.882060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.882464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.882464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.882867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.883828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.884450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.884657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.884873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.885091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.885295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.885513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.885733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.885940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.886144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.886366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.886569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.886805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.887010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.887267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.887493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.887700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.887918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.888143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.888143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.888533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.888735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.888735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.889154 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.889374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.889593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.889800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.890014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.890221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.890221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.890620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.890824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.891025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.891324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.891615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.891901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.892124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.892333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.892547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.892751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.892751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.893158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.893374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.893374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.893769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.893983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.894206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.894412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.894613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.894830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.895063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.895287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.895513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.895729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.895933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.896155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.896360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.896578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.896785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.897000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.897000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.897414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.897631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.897848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.898055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.898258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.898486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.898693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.898897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.899143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.899363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.899583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.899800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.900006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.900245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.900245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.900651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.900868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.901074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.901280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.901500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.901706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.901919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.902137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.902340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.902544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.902786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.903004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.903358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.903566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.903566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.903973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.904296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.904503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.904710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.904914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.905120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.905374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.905599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.905823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.906034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.906239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.906454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.906454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.906881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.907120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.907380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.907591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.907811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.908055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.908275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.908513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.908740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.909267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.909267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.909720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.909940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.910168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.910398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.910625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.910856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.910856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.911313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.911547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.911789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.912005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.912240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.912466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.912694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.912924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.913152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.913378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.913605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.913831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.914048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.914265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.914490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.914734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.914968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.915219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.915438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.915654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.915887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.916116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.916332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.917254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.917737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.917963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.918197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.918452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.918672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.918889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.918889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.919387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.919616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.919843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.920710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.922385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.924179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.931967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.935018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.936757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.938432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.940215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.941864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.943614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.945300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.946974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.948673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.950576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.952415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.954059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.955856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.957540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 14:47:01.959356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=124397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 14:47:02.072437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO dbcounter [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Registered counter for database nova_api Jul 20 14:47:02.095324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_db.sqlalchemy.engines [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.104520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writer thread running {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.119001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO dbcounter [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Registered counter for database nova_api Jul 20 14:47:02.139298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_db.sqlalchemy.engines [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.147039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writer thread running {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.163639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.context [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f84c0e38-8de4-44d7-ae90-44f10e366750(cell1) {{(pid=124396) load_cells /opt/stack/nova/nova/context.py:464}} Jul 20 14:47:02.167896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:47:02.169515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:47:02.170329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:47:02.172366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:47:02.174519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO dbcounter [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Registered counter for database nova_cell0 Jul 20 14:47:02.178579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO dbcounter [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Registered counter for database nova_cell1 Jul 20 14:47:02.188328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_db.sqlalchemy.engines [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.189992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_db.sqlalchemy.engines [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.193281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writer thread running {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.204184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.context [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),f84c0e38-8de4-44d7-ae90-44f10e366750(cell1) {{(pid=124397) load_cells /opt/stack/nova/nova/context.py:464}} Jul 20 14:47:02.208404 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:47:02.209993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:47:02.210810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:47:02.212838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:47:02.214836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO dbcounter [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Registered counter for database nova_cell0 Jul 20 14:47:02.218896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO dbcounter [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Registered counter for database nova_cell1 Jul 20 14:47:02.227620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_db.sqlalchemy.engines [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.230223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_db.sqlalchemy.engines [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.235332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writer thread running {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.241362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writer thread running {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.334843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writer thread running {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.345287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO dbcounter [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Registered counter for database nova_cell0 Jul 20 14:47:02.360965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_db.sqlalchemy.engines [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.366412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writer thread running {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.392159 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO dbcounter [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Registered counter for database nova_cell0 Jul 20 14:47:02.399362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_db.sqlalchemy.engines [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 14:47:02.406353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writer thread running {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 14:47:02.682263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: WARNING keystonemiddleware.auth_token [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:47:02.689469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: WARNING keystonemiddleware.auth_token [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:47:02.721264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: WARNING keystonemiddleware.auth_token [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:47:02.728607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: WARNING keystonemiddleware.auth_token [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:47:03.027435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: WARNING keystonemiddleware.auth_token [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:47:03.033361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: WARNING keystonemiddleware.auth_token [None req-323ab513-ed9b-4f22-8482-10e7dcdc5f45 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:47:03.039824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a17a6399c0 pid: 124396 (default app) Jul 20 14:47:03.064101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: WARNING keystonemiddleware.auth_token [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 20 14:47:03.070084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: WARNING keystonemiddleware.auth_token [None req-2d0ba9fa-ca9a-4522-a385-0dba740b864c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 20 14:47:03.076291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a17a6399c0 pid: 124397 (default app) Jul 20 14:47:45.686598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.688537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.689973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:45.691670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:47.159024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7740b49e-597a-44db-99bc-a05fdeb3de50 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Action: 'create', calling method: .version_select at 0x7f7b1847ec10>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-164175742", "metadata": {"Type": "work"}}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7740b49e-597a-44db-99bc-a05fdeb3de50 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 121 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:47:47.195939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-164175742", "metadata": {"Type": "work"}}} Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:47:47.197472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-164175742', 'metadata': {'Type': 'work'}}}} Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise error Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi '#$%' Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:47.198350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 20 14:47:47.199816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 20 14:47:47.201071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 20 14:47:47.201071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.201071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7740b49e-597a-44db-99bc-a05fdeb3de50 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:47.201071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7740b49e-597a-44db-99bc-a05fdeb3de50 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.528866 Jul 20 14:47:47.208777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1/1] 10.222.0.15 () {60 vars in 1155 bytes} [Wed Jul 20 14:47:45 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1538 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 14:47:47.215510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:47.219036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:47.225022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:47.226906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:47:47.632182 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d32faaa-4951-44e3-9c0a-04639330e5bf tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Action: 'create', calling method: .version_select at 0x7f7b184d0700>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-846499764", "metadata": {"Type": "work"}}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d32faaa-4951-44e3-9c0a-04639330e5bf tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 119 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:47:47.642309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-846499764", "metadata": {"Type": "work"}}} Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: Jul 20 14:47:47.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-846499764', 'metadata': {'Type': 'work'}}}} Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise error Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi '0' Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:47.644979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 20 14:47:47.646731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 20 14:47:47.648273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 20 14:47:47.648273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.660685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d32faaa-4951-44e3-9c0a-04639330e5bf tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:47.662782 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1d32faaa-4951-44e3-9c0a-04639330e5bf tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.454800 Jul 20 14:47:47.665196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 1/2] 10.222.0.15 () {60 vars in 1155 bytes} [Wed Jul 20 14:47:47 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 459 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 14:47:47.675079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0bb7349e-bf6f-4495-9159-426cb06f814a tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Action: 'create', calling method: .version_select at 0x7f7b183daee0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1437742825", "metadata": {"Type": "work"}}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0bb7349e-bf6f-4495-9159-426cb06f814a tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 119 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:47:47.678635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1437742825", "metadata": {"Type": "work"}}} Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:47:47.679948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1437742825', 'metadata': {'Type': 'work'}}}} Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise error Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi '' Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:47.680705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 20 14:47:47.681962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 20 14:47:47.683258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 20 14:47:47.683258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:47.692188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0bb7349e-bf6f-4495-9159-426cb06f814a tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:47.693882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0bb7349e-bf6f-4495-9159-426cb06f814a tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.023369 Jul 20 14:47:47.694726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2/3] 10.222.0.15 () {60 vars in 1155 bytes} [Wed Jul 20 14:47:47 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 14:47:47.722323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec353c8d-e2ff-4762-815c-accf2f762aa1 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Calling method '.version_select at 0x7f7b1844d940>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:48.281907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-ec353c8d-e2ff-4762-815c-accf2f762aa1 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] HTTP exception thrown: Volume tempest-invalid-1758450370 could not be found. Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec353c8d-e2ff-4762-815c-accf2f762aa1 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1758450370 HTTP/1.1 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:47:48.287557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Volume tempest-invalid-1758450370 could not be found. Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': 'tempest-invalid-1758450370'} Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:48.288969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1758450370 could not be found. (HTTP 404) (Request-ID: req-dc345a7e-b0a1-4fd8-9ea4-8422196f3153) Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.289865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:47:48.291207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1758450370 could not be found. Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:47:48.292297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1758450370 could not be found. Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1758450370 could not be found. Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec353c8d-e2ff-4762-815c-accf2f762aa1 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 404 to user: Volume tempest-invalid-1758450370 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ec353c8d-e2ff-4762-815c-accf2f762aa1 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1758450370" status: 404 len: 99 microversion: 2.1 time: 0.588628 Jul 20 14:47:48.293588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1217 bytes} [Wed Jul 20 14:47:47 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1758450370 => generated 99 bytes in 593 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 20 14:47:48.301849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36527b84-7d21-491b-9fc1-8896a02e0501 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004868 Jul 20 14:47:48.311474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 3/5] 10.222.0.15 () {58 vars in 1139 bytes} [Wed Jul 20 14:47:48 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 15 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 20 14:47:48.317323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3bb7b5f8-acdf-415a-b441-a76ca0175024 tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004262 Jul 20 14:47:48.317992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 3/6] 10.222.0.15 () {58 vars in 1136 bytes} [Wed Jul 20 14:47:48 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 20 14:47:48.328635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cafe674e-a92f-422a-b07d-ec91a6f374ab tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Calling method '.version_select at 0x7f7b1847ed30>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:48.590719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-cafe674e-a92f-422a-b07d-ec91a6f374ab tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] HTTP exception thrown: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cafe674e-a92f-422a-b07d-ec91a6f374ab tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/os-volumes/ef212ba6-30cb-42d5-9ce8-608b64f1e69c HTTP/1.1 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:47:48.601229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'ef212ba6-30cb-42d5-9ce8-608b64f1e69c'} Jul 20 14:47:48.602640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. (HTTP 404) (Request-ID: req-f31d51ad-1c08-4cb0-9101-4e69e08d5231) Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.603712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:47:48.604917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 20 14:47:48.606041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. Jul 20 14:47:48.607433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.611007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cafe674e-a92f-422a-b07d-ec91a6f374ab tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 404 to user: Volume ef212ba6-30cb-42d5-9ce8-608b64f1e69c could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:48.612316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cafe674e-a92f-422a-b07d-ec91a6f374ab tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/ef212ba6-30cb-42d5-9ce8-608b64f1e69c" status: 404 len: 109 microversion: 2.1 time: 0.289011 Jul 20 14:47:48.614103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:47:48 2022] DELETE /compute/v2.1/os-volumes/ef212ba6-30cb-42d5-9ce8-608b64f1e69c => generated 109 bytes in 291 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:47:48.623730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-932cf6e4-05a3-4a2e-a413-33c305eae50b tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Calling method '.version_select at 0x7f7b182da700>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:48.678607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-932cf6e4-05a3-4a2e-a413-33c305eae50b tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] HTTP exception thrown: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-932cf6e4-05a3-4a2e-a413-33c305eae50b tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Request method failure captured: Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/os-volumes/0bc876e3-b089-498a-99db-a86fc1153b6e HTTP/1.1 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWRe0dNu1VI9msV_kC8zjRJ4gtec3uI2_OGC5zpWN_F2-5ZxSSSoQbul-sQucRFpBpn3q84gBLN7t_rr9YfuSIkthCZ9b9uZfW_6fUmgmliwPhC9b1AcajJS9qiMd6OjNgTB5hVm6pPRlSf_8DRbiIoK_IKc6ijiz0vp7Q7ehogys4 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:47:48.680797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/70bb0d5b8f4440e9b5662cb4a7cbaad9"}]}] Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 70bb0d5b8f4440e9b5662cb4a7cbaad9 Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesNegativeTest-52473330 Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 99097185f6744d778063374e18c9dca2 Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesNegativeTest-52473330-project Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '0bc876e3-b089-498a-99db-a86fc1153b6e'} Jul 20 14:47:48.682193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. (HTTP 404) (Request-ID: req-9625c504-b049-48cb-b7a9-7aecba9a6c43) Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.683106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:47:48.684238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 20 14:47:48.685468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. Jul 20 14:47:48.686718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:47:48.690656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-932cf6e4-05a3-4a2e-a413-33c305eae50b tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] Returning 404 to user: Volume 0bc876e3-b089-498a-99db-a86fc1153b6e could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:47:48.692377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-932cf6e4-05a3-4a2e-a413-33c305eae50b tempest-VolumesNegativeTest-52473330 tempest-VolumesNegativeTest-52473330-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/0bc876e3-b089-498a-99db-a86fc1153b6e" status: 404 len: 109 microversion: 2.1 time: 0.071968 Jul 20 14:47:48.695268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:47:48 2022] GET /compute/v2.1/os-volumes/0bc876e3-b089-498a-99db-a86fc1153b6e => generated 109 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:47:49.317271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1eca092d-8300-454a-9fc3-9de142fa0aca tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b183dae50>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:49.442103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1eca092d-8300-454a-9fc3-9de142fa0aca tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/944006ff-16d3-44d9-9f18-3c778a54a7d3" status: 200 len: 350 microversion: 2.1 time: 0.127285 Jul 20 14:47:49.442738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 5/9] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:47:49 2022] GET /compute/v2.1/os-volumes/944006ff-16d3-44d9-9f18-3c778a54a7d3 => generated 350 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:51.455504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2731b67a-9f42-413f-afbe-8af7fc4dece6 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182da670>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:51.907207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2731b67a-9f42-413f-afbe-8af7fc4dece6 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/bbb424da-866d-4466-ad0f-d40c6ec353e3" status: 200 len: 350 microversion: 2.1 time: 0.456567 Jul 20 14:47:51.908094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 5/10] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:47:51 2022] GET /compute/v2.1/os-volumes/bbb424da-866d-4466-ad0f-d40c6ec353e3 => generated 350 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:54.205093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c2ecb69-ef33-442f-aa1f-dd97374cee33 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182ceca0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:54.355981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c2ecb69-ef33-442f-aa1f-dd97374cee33 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/66017250-5312-46ba-b4ef-205e6cdfc08f" status: 200 len: 349 microversion: 2.1 time: 0.154890 Jul 20 14:47:54.356564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 6/11] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:47:54 2022] GET /compute/v2.1/os-volumes/66017250-5312-46ba-b4ef-205e6cdfc08f => generated 349 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:47:54.368980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4cba8982-67d3-43af-a30c-752348175da9 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182daca0>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:54.591557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4cba8982-67d3-43af-a30c-752348175da9 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes" status: 200 len: 1032 microversion: 2.1 time: 0.226796 Jul 20 14:47:54.592440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1133 bytes} [Wed Jul 20 14:47:54 2022] GET /compute/v2.1/os-volumes => generated 1032 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:47:54.607347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c1315146-9801-4fd2-9f07-d601b3bd9436 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182e63a0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:54.856704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c1315146-9801-4fd2-9f07-d601b3bd9436 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.246502 Jul 20 14:47:54.856704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1148 bytes} [Wed Jul 20 14:47:54 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 692 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:54.862090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-13b2fac4-e2cd-4b40-a098-5be222854038 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b184d0040>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:55.017293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-13b2fac4-e2cd-4b40-a098-5be222854038 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes" status: 200 len: 1032 microversion: 2.1 time: 0.159175 Jul 20 14:47:55.017969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 7/14] 10.222.0.15 () {58 vars in 1133 bytes} [Wed Jul 20 14:47:54 2022] GET /compute/v2.1/os-volumes => generated 1032 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:47:55.029581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35d50a8a-4e9c-4b8e-9468-a063a8b3b4ed tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182eb040>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:55.246395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-35d50a8a-4e9c-4b8e-9468-a063a8b3b4ed tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.220812 Jul 20 14:47:55.248058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 8/15] 10.222.0.15 () {58 vars in 1166 bytes} [Wed Jul 20 14:47:55 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:55.275557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-20d9e650-dbe2-402f-9af4-42eab732749f tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182da940>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:55.501317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-20d9e650-dbe2-402f-9af4-42eab732749f tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 692 microversion: 2.1 time: 0.234713 Jul 20 14:47:55.502451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1169 bytes} [Wed Jul 20 14:47:55 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 692 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:55.516705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a85e1387-487b-4b33-bfc1-71386871d678 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182e64c0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:55.666115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a85e1387-487b-4b33-bfc1-71386871d678 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1032 microversion: 2.1 time: 0.153531 Jul 20 14:47:55.668047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 9/17] 10.222.0.15 () {58 vars in 1154 bytes} [Wed Jul 20 14:47:55 2022] GET /compute/v2.1/os-volumes/detail => generated 1032 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:47:55.680261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e2b95259-ede0-44c7-90f2-6770b632892d tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182df9d0>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:55.869649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e2b95259-ede0-44c7-90f2-6770b632892d tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.194360 Jul 20 14:47:55.874377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 9/18] 10.222.0.15 () {58 vars in 1187 bytes} [Wed Jul 20 14:47:55 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:47:55.889454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9af2b813-9117-4464-adda-9a221cb99552 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] Calling method '.version_select at 0x7f7b182cef70>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:47:56.066778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9af2b813-9117-4464-adda-9a221cb99552 tempest-VolumesTestJSON-1820422123 tempest-VolumesTestJSON-1820422123-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1032 microversion: 2.1 time: 0.186316 Jul 20 14:47:56.067680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 10/19] 10.222.0.15 () {58 vars in 1154 bytes} [Wed Jul 20 14:47:55 2022] GET /compute/v2.1/os-volumes/detail => generated 1032 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:00.511266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-975999397", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "networks": [{"uuid": "0f8c175d-a2fb-4121-97df-865f9696c92f"}]}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:00.796694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.network.neutron [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] validate_networks() for [('0f8c175d-a2fb-4121-97df-865f9696c92f', None, None, None, None, None)] {{(pid=124397) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:03.365175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63717b9d-db61-41cf-a3d2-7f03bf4527f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: .version_select at 0x7f7b18cdd0d0>, body: {"keypair": {"name": "tempest-keypair-1075877835", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFfkiDB15WwDcsBn/0v262I9qIWZq8nJfndunwL+rHxLwNLlkr2z0D1ZfPRD3PEtOgmYrOTcyx+yNZGMTY63ywSZec0Ds+Ih+vWC69hWQvihNZMVAekEY+jYHj+pfIxzGA=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:03.539265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-63717b9d-db61-41cf-a3d2-7f03bf4527f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:03.563206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-63717b9d-db61-41cf-a3d2-7f03bf4527f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:03.678361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63717b9d-db61-41cf-a3d2-7f03bf4527f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.701376 Jul 20 14:48:03.685810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 11/20] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:48:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:03.985808 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Flavor limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:03.986341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Image limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:03.986685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Flavor pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:03.987351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Image pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:03.988084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:04.023002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:04.023841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:04.029963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:04.030654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:04.135569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Getting quotas for project 3d28cd25ee0c476e8ab3b24d7f2f9ab9. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:04.168168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Getting quotas for user 5373c0299ea64c9f872e3bc04bfb0b74 and project 3d28cd25ee0c476e8ab3b24d7f2f9ab9. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:04.204798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Going to run 1 instances... {{(pid=124397) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:04.208763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124397) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:04.413753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-59f2fc3d-e63f-475b-8d37-d1207098d020 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.694717 Jul 20 14:48:04.416010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 10/21] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:47:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 4698 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:48:04.432757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8c218b20-c81c-48fb-ab1b-23dc40bb2722 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:04.434030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8c218b20-c81c-48fb-ab1b-23dc40bb2722 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:04.874622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fdfee472-4ee8-449a-92bb-0bd9c0d083f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: .version_select at 0x7f7b182afca0>, body: {"keypair": {"name": "tempest-keypair-1676120478", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBILBJcrdMmTSVIR1W0OegHTGwFGNaGxnjsdbTRamJseaCdmeK8yoaAXEvg4Lno4/w7VX0ePSV/MgM/Ai2W5/DmweriBLN48E7epRx+fNXiC7TIrsIUuX/y1rJ0XrS0mbKQ=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:04.883764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8c218b20-c81c-48fb-ab1b-23dc40bb2722 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:04.896383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-fdfee472-4ee8-449a-92bb-0bd9c0d083f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:04.912368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-fdfee472-4ee8-449a-92bb-0bd9c0d083f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for user ee5a234904e64eb9b2faad85909d0507 and project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:04.927951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8c218b20-c81c-48fb-ab1b-23dc40bb2722 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1214 microversion: 2.1 time: 0.503197 Jul 20 14:48:04.929827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 11/22] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:04 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1214 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:04.987836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fdfee472-4ee8-449a-92bb-0bd9c0d083f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.825472 Jul 20 14:48:04.991768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 12/23] 10.222.0.15 () {62 vars in 1199 bytes} [Wed Jul 20 14:48:04 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 831 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Jul 20 14:48:05.945773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d8d0b895-7820-45ec-8ed9-1634dea621cb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:05.946389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d8d0b895-7820-45ec-8ed9-1634dea621cb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:06.168545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d8d0b895-7820-45ec-8ed9-1634dea621cb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:06.181386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d8d0b895-7820-45ec-8ed9-1634dea621cb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1214 microversion: 2.1 time: 0.237444 Jul 20 14:48:06.200069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:05 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1214 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:06.986039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2021051518", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:07.160638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] validate_networks() for [('ce763c04-6208-4594-840c-20615d6164a4', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:07.198525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:07.200637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:07.216829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:07.217279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:07.218059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:07.218528 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:07.815229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:07.848377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b7c66101-173f-4f4f-98e5-a3710005afae tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1210 microversion: 2.1 time: 0.646636 Jul 20 14:48:07.850893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 13/25] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:07 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1210 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:08.044039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d380e1e9-1338-4d49-839e-38a7aa473d05 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: .version_select at 0x7f7b17102790>, body: {"keypair": {"name": "tempest-keypair-2080322666", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFn1vAJjdsxarR8tP5SWd5KTDSxADnqOvqSmtisrpEDXG/ysmnwBoTq57v7g05jbkJkQFfPpiP1vFE1oKfEH7DMpl5TC1xScwO+bWDnPJkUJOZhpvHArlWFC4rk1led/Vw=="}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:08.084292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-d380e1e9-1338-4d49-839e-38a7aa473d05 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for project b966dbac18614c979f674666ba817def. Resources: {'key_pairs'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:08.095953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-d380e1e9-1338-4d49-839e-38a7aa473d05 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for user 5f51124af1e846fb89552078c81afdb0 and project b966dbac18614c979f674666ba817def. Resources: {'key_pairs'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:08.194082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d380e1e9-1338-4d49-839e-38a7aa473d05 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.340683 Jul 20 14:48:08.200324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 14/26] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:48:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:08.868580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:08.869709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:08.898989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:08.906825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:08.908763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:08.912389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:09.134193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:09.167260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5b0ec5ff-e2d0-4083-b2cf-2fa56722e83c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1284 microversion: 2.1 time: 0.303540 Jul 20 14:48:09.172678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 15/27] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:08 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1284 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:10.193007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:10.194375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:10.208749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:10.214632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:10.216402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:10.217013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:11.577875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:11.577875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3972109a-6eec-41f6-91c8-ff331273d977 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1272 microversion: 2.1 time: 1.386954 Jul 20 14:48:11.589512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 16/28] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:10 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1272 bytes in 1402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:11.606917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-792405064", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "bb36c589-f60d-4c91-8345-5777287e4f0d"}], "security_groups": [{"name": "tempest-securitygroup--919259833"}], "key_name": "tempest-keypair-1075877835", "user_data": "}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:12.298708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:12.299290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:12.299625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:12.300033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:12.300450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:12.338858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.network.neutron [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] validate_networks() for [('bb36c589-f60d-4c91-8345-5777287e4f0d', None, None, None, None, None)] {{(pid=124397) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:12.346662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:12.347089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:12.347997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:12.348438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:12.498712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:12.522752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for user 938083ff74d84224be142a26d8e17ad6 and project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:12.613093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:12.629291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:12.957298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e150dc4e-9711-4d8e-a479-c0dccd19d239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 6.581201 Jul 20 14:48:12.958601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 13/29] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:48:06 2022] POST /compute/v2.1/servers => generated 383 bytes in 6584 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:48:12.999674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1267248355", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4a7c71d7-d29f-4122-81d5-864e01fddc6d"}], "security_groups": [{"name": "tempest-securitygroup--670912933"}], "key_name": "tempest-keypair-1676120478", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:14.071086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:14.076479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] validate_networks() for [('4a7c71d7-d29f-4122-81d5-864e01fddc6d', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:15.487345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:15.492025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:15.492730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:15.493677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:15.494894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:15.519486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:15.520263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:15.521862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:15.522510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:15.580628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:15.585554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:15.668659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Going to run 1 instances... {{(pid=124397) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:15.671117 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124397) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:15.772071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1f07a65-ba43-4ea2-8ca7-4b6742fafb66 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 4.181059 Jul 20 14:48:15.775825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 17/30] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:48:11 2022] POST /compute/v2.1/servers => generated 401 bytes in 4185 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:48:15.792167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:15.799387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:15.833727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:15.836826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:15.837875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:15.839419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:15.993428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:15.994535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:15.995073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:15.995831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:15.996408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:16.028909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:16.029372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:16.030179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:16.030863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:16.096524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:16.105635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for user ee5a234904e64eb9b2faad85909d0507 and project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:16.139455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:16.142542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:16.243424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5d228640-640d-4771-b015-ed6f09b4e906 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 3.273474 Jul 20 14:48:16.252623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 14/31] 10.222.0.15 () {62 vars in 1187 bytes} [Wed Jul 20 14:48:12 2022] POST /compute/v2.1/servers => generated 401 bytes in 3287 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 20 14:48:16.260001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:16.267739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:16.328816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:16.346428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:16.352342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:16.355901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:16.915519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:16.931394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-acdea4e3-8247-436f-bd59-111f3ddec83e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 1.151676 Jul 20 14:48:16.941518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 18/32] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:15 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 1162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:16.975133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f80bb319-9eda-4e34-bd6b-e84ebade8fa2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:16.976091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f80bb319-9eda-4e34-bd6b-e84ebade8fa2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:17.346491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:17.388840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3661084b-5bcd-4578-8813-ed52174ce180 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1282 microversion: 2.1 time: 1.134915 Jul 20 14:48:17.401756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 15/33] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:16 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1282 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:17.431441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:17.432643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:17.750655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f80bb319-9eda-4e34-bd6b-e84ebade8fa2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:17.789300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f80bb319-9eda-4e34-bd6b-e84ebade8fa2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1241 microversion: 2.1 time: 0.844161 Jul 20 14:48:17.792556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 19/34] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1241 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:17.917221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1661857943", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:17.973430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:18.028458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:18.030292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:18.031667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-635a0e31-7814-4e35-9424-fe16e48c8653 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1282 microversion: 2.20 time: 0.628770 Jul 20 14:48:18.032560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 16/35] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1282 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:18.084467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:18.087818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:18.106695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.network.neutron [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] validate_networks() for [('ce763c04-6208-4594-840c-20615d6164a4', None, None, None, None, None)] {{(pid=124397) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:18.110899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:18.116870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:18.121004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:18.145844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:19.211186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:19.234574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff3e524b-c31a-4e4a-9f40-7cb87fc2ae22 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 1.200766 Jul 20 14:48:19.235384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 17/36] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:18 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 1202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:19.247155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:19.247916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:19.279742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:19.283671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:19.285823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:19.286341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:19.560516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:19.593285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f32454c6-e4da-419c-8eca-2f5de283f630 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1293 microversion: 2.1 time: 0.355883 Jul 20 14:48:19.595199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 18/37] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1293 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:19.600591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:19.611443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:19.645119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:19.656710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:19.657984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:19.658443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:20.076127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:20.110668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:20.125225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:20.149175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cf5c91e4-9731-496f-ae6e-2eae26a451a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1334 microversion: 2.20 time: 0.532842 Jul 20 14:48:20.159485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 19/38] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1334 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:20.618358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-798187e3-e446-4b01-8890-89dc5d289ffd req-599a2b06-0d68-4042-bc24-1b5bfe09b17e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ad8269f-e328-4587-a622-d5940cd92fa7", "tag": "e938812d-42ef-47c6-9b5a-d4cab1fca9f5"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:20.669246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-798187e3-e446-4b01-8890-89dc5d289ffd req-599a2b06-0d68-4042-bc24-1b5bfe09b17e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:20.669702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-798187e3-e446-4b01-8890-89dc5d289ffd req-599a2b06-0d68-4042-bc24-1b5bfe09b17e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:20.754394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-798187e3-e446-4b01-8890-89dc5d289ffd req-599a2b06-0d68-4042-bc24-1b5bfe09b17e service nova] Creating event network-changed:e938812d-42ef-47c6-9b5a-d4cab1fca9f5 for instance 5ad8269f-e328-4587-a622-d5940cd92fa7 on hv-cinder-84902 Jul 20 14:48:20.755257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-798187e3-e446-4b01-8890-89dc5d289ffd req-599a2b06-0d68-4042-bc24-1b5bfe09b17e service nova] Acquired lock "compute-rpcapi-router" {{(pid=124396) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 20 14:48:20.757231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:20.757855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:20.760858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:20.761663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:20.800927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.compute.rpcapi [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 20 14:48:20.802238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] Releasing lock "compute-rpcapi-router" {{(pid=124396) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 20 14:48:20.850699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5126b6d0-59c0-4edf-8a43-46cd0c7de090 None None] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.630604 Jul 20 14:48:20.851782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 20/39] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:48:20.858996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:20.860356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:20.881947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:20.882404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:20.891367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:20.891985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:21.112934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:21.113432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:21.113823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:21.114399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:21.114969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:21.153264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:21.154546 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:21.157896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:21.162098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:21.234943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:21.250205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for user 938083ff74d84224be142a26d8e17ad6 and project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:21.299290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Going to run 1 instances... {{(pid=124397) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:21.306740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124397) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:21.449009 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-87961912-b262-49d4-91a4-ad263ee0e177 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.536254 Jul 20 14:48:21.453703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 20/40] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:48:17 2022] POST /compute/v2.1/servers => generated 383 bytes in 3540 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 20 14:48:21.465738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:21.466768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:21.533271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:21.533720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:21.536384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:21.536966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:22.000705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.027523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fb4b984b-a79e-430c-9a55-794e2a1004e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1299 microversion: 2.1 time: 0.572339 Jul 20 14:48:22.030749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 21/41] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1299 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:22.045665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:22.051387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:22.079245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:22.082978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:22.096575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:22.101095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:22.434625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.446273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.450551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.453201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d2ce5d21-281d-4e34-ab2f-231304e54824 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1340 microversion: 2.20 time: 0.421225 Jul 20 14:48:22.455863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 22/42] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1340 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:22.471186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:22.473934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:22.506976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:22.518561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:22.521642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:22.524196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:22.544578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.571681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff8092f0-04a5-44b3-8d83-8a9f4f8ace91 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 1.715005 Jul 20 14:48:22.588504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 21/43] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:20 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 1736 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:22.606555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1617174984", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "696bc9e1-cb60-427e-98c0-b21d1bdfb8d1"}], "security_groups": [{"name": "tempest-securitygroup--382298835"}], "key_name": "tempest-keypair-2080322666", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:22.872945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:22.885836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5477456f-b930-4b39-b2e9-4375f3537fa4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1226 microversion: 2.1 time: 0.428588 Jul 20 14:48:22.897812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 23/44] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:22 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1226 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:23.040186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] validate_networks() for [('696bc9e1-cb60-427e-98c0-b21d1bdfb8d1', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:48:23.047359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:23.048123 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:23.064049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:23.068249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:23.069302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:23.071666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:23.708314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:23.750265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-28bcbcd1-f9c7-4fe2-b4d8-0c615b06b556 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1299 microversion: 2.1 time: 0.706910 Jul 20 14:48:23.753821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 24/45] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1299 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:23.758737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:23.783691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:23.802826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:23.810935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:23.812931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:23.814058 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:24.532838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:24.559490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:24.577745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:24.584152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-be6444dc-35ac-4415-a482-e48b8a6dbf35 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1340 microversion: 2.20 time: 0.825808 Jul 20 14:48:24.587436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 25/46] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1340 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:24.612647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:24.614547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:24.644247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:24.645092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:24.670214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:24.670214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:25.458314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:25.494862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-372d2945-e9a6-4e39-bc2f-9372135c0ebb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.904643 Jul 20 14:48:25.504112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 26/47] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:24 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:25.512992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:25.518621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:25.530856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:25.534982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:25.536920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:25.539083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:25.755926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:25.767464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-96bb97f1-8396-4ff9-b989-6b6c5b755b4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1280 microversion: 2.1 time: 0.261903 Jul 20 14:48:25.768216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 27/48] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:25 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1280 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:25.793597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:25.794642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:25.810891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:25.818009 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:25.818840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:25.819308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.467441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:48:26.468546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:48:26.469391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:48:26.469962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:48:26.470580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:48:26.473542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:26.541766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=23 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:26.556596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.557090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.558098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.558529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.622794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for project b966dbac18614c979f674666ba817def. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:26.631965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for user 5f51124af1e846fb89552078c81afdb0 and project b966dbac18614c979f674666ba817def. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:26.668154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:48:26.673759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:48:26.782486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c96ac652-c484-44e3-af0e-b8228269f798 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 4.191522 Jul 20 14:48:26.784713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 22/49] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:48:22 2022] POST /compute/v2.1/servers => generated 401 bytes in 4195 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:48:26.795974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:26.797417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:26.804035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:26.815675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.816762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.852897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-32998752-b685-4c42-a4f8-aee3c9ccda8a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 1.079209 Jul 20 14:48:26.856725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.857265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.879396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 28/50] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 1107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:26.900769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:26.904371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:26.910054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:26.934248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.935150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:26.936232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:26.937620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:27.346610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:27.363753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:27.369517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:27.372417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16d58c28-1e06-4b9e-9590-4f52ea2a4c08 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1340 microversion: 2.20 time: 0.585565 Jul 20 14:48:27.373029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 23/51] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1340 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:27.405341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:27.407801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:27.430504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:27.432887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:27.440392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:27.441394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.076503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:28.086812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4e48dd88-9101-4fba-86c7-24594dcf1e77 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 1.204387 Jul 20 14:48:28.089847 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 29/52] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:26 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 1209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:28.120710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:28.121671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:28.142429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.151789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.152883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.156236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.472555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:28.510439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63695f0f-4ee5-4eee-8aee-224c5b0816b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 1.123895 Jul 20 14:48:28.510796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:28.511068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 24/53] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:27 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 1137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:28.545954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:28.546709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5e62b158-79a4-480f-8329-8d8cb8ffcccd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1242 microversion: 2.1 time: 0.447460 Jul 20 14:48:28.548053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:28.548200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 30/54] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1242 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:28.560579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.561297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.562369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-df693faa-bcb0-4a78-ae3c-b33e19cec857 req-ad40f909-d6a1-4915-9061-18b13be85565 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6b566c6c-a418-4384-b99e-c7295e754e92", "tag": "67324695-43bc-444a-9a35-3a3bbb87de0b"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:28.565686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.568084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.602595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-df693faa-bcb0-4a78-ae3c-b33e19cec857 req-ad40f909-d6a1-4915-9061-18b13be85565 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.603466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-df693faa-bcb0-4a78-ae3c-b33e19cec857 req-ad40f909-d6a1-4915-9061-18b13be85565 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.658894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-df693faa-bcb0-4a78-ae3c-b33e19cec857 req-ad40f909-d6a1-4915-9061-18b13be85565 service nova] Creating event network-changed:67324695-43bc-444a-9a35-3a3bbb87de0b for instance 6b566c6c-a418-4384-b99e-c7295e754e92 on hv-cinder-84902 Jul 20 14:48:28.659846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-df693faa-bcb0-4a78-ae3c-b33e19cec857 req-ad40f909-d6a1-4915-9061-18b13be85565 service nova] Acquired lock "compute-rpcapi-router" {{(pid=124397) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 20 14:48:28.663210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.665450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.666531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.669691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.688057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.compute.rpcapi [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 20 14:48:28.689196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] Releasing lock "compute-rpcapi-router" {{(pid=124397) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 20 14:48:28.713509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-acea38cc-aa95-42b7-9ce1-da4e420e5af6 None None] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.163285 Jul 20 14:48:28.715407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 31/55] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:28.723832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:28.725655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:28.745948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.746867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.748254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:28.749135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:28.973471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:28.992447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8519b405-2e03-4210-9013-b7fd9f2e9c92 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.475514 Jul 20 14:48:28.994004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 25/56] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:29.112363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:29.113118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:29.122776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.131919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.133022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.133624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.156278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.160439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:29.169835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.178425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.179900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d6587188-3e08-4559-9242-21f1cc43f507 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.461390 Jul 20 14:48:29.181039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 32/57] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:29.422533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.436688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f8bcd734-6a04-4be4-a26a-2484eec00801 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.328280 Jul 20 14:48:29.437294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 26/58] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:29 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:29.540750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:29.541916 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:29.556671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.557415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.568914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:29.571297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:29.575953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.578250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.590122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.593810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.598377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:29.598925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:29.833188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.841926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59de632e-56b4-4fe7-a9d3-2506cc0dff6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1242 microversion: 2.1 time: 0.280604 Jul 20 14:48:29.845659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 27/59] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1242 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:29.886580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:29.895471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-29f7446a-2f1a-4328-a885-af8bc0d4fb8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.368030 Jul 20 14:48:29.896022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 33/60] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:29 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:30.018996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:30.020060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:30.039098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.039877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.041121 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.041853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.214433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-6b6b0e8f-733f-4bea-8a60-f4fcd29555dd req-98b0421f-6fc1-44c2-b9f5-43ccb48300f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "304148dc-eb2a-4dec-a9ec-1cff5c5114b7", "tag": "c215e006-b33e-46f8-be1f-7cc86ace4f37"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:30.225249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6b6b0e8f-733f-4bea-8a60-f4fcd29555dd req-98b0421f-6fc1-44c2-b9f5-43ccb48300f0 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.226232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6b6b0e8f-733f-4bea-8a60-f4fcd29555dd req-98b0421f-6fc1-44c2-b9f5-43ccb48300f0 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.247417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-6b6b0e8f-733f-4bea-8a60-f4fcd29555dd req-98b0421f-6fc1-44c2-b9f5-43ccb48300f0 service nova] Creating event network-changed:c215e006-b33e-46f8-be1f-7cc86ace4f37 for instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 on hv-cinder-84902 Jul 20 14:48:30.253685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-6b6b0e8f-733f-4bea-8a60-f4fcd29555dd req-98b0421f-6fc1-44c2-b9f5-43ccb48300f0 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.308580 Jul 20 14:48:30.255007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 28/61] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:30.261069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:30.261784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:30.269602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.270175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.271097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.271660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.447331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.458342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b3da9fe3-b2bc-4c26-b5cd-1036199c1f38 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.445812 Jul 20 14:48:30.458856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 34/62] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:30 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:30.477117 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:30.478418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:30.487932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.488761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.490782 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.491715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.520040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.530384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.533030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.534592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad1f11c6-11d1-47ab-999c-c4824a32a8f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.278118 Jul 20 14:48:30.535462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 29/63] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:30.680495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.693062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e8f2789d-d36f-44fe-9bb9-de60b4f98ac5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.221764 Jul 20 14:48:30.694385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 35/64] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:30 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:30.870515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:30.872669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:30.885001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.885830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.887420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.888212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.919189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:30.920234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:30.936298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.937244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.938653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:30.939512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:30.989047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:30.998220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86c46f81-6870-4031-883f-aa2c8a2bc4b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1242 microversion: 2.1 time: 0.133455 Jul 20 14:48:31.019478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 30/65] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1242 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:31.107647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:31.116902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-27541f5e-80ee-443c-a969-95e69ec13461 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.206508 Jul 20 14:48:31.117534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 36/66] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:30 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:31.477643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:31.479311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:31.493042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.493515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.495476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.495944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.558364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:31.559303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:31.563313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=28 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:31.569581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.570417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.572242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.572873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.717343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:31.730433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0f97bcf-e1c5-4310-8746-7f5f2c52d2f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.257366 Jul 20 14:48:31.731523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 31/67] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:31.738996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:31.739861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:31.747744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.748199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.749465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:31.750400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:31.795640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:31.806247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:31.808265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:31.809788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5676cfb5-8ef4-44cf-8066-076ee008614e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.257121 Jul 20 14:48:31.810958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 37/68] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:32.017885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:32.018501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:32.043444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.047529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.049509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.052983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.074730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:32.084017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d68466b6-7d89-445c-9c7a-9821cda29a80 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.351178 Jul 20 14:48:32.093765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 32/69] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:31 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:32.144491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:32.145423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:32.155964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.157091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.164352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.164822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.228065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:32.271400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:32.287909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-166d83b3-1e6d-4540-ab77-45db459f84af tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1234 microversion: 2.1 time: 0.274212 Jul 20 14:48:32.296328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 38/70] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1234 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:32.503882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:32.515678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d387f52a-5749-4499-8f7f-237eb350ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.376526 Jul 20 14:48:32.516260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 33/71] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:32 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:32.748189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:32.748877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:32.755957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.756439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.757739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.758447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.841198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:32.850260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:32.880261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.880261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:32.880261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:32.880261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.235160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.249665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ef284716-3cc6-4e5d-8feb-b68f7b97117c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.504747 Jul 20 14:48:33.251017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 39/72] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:33.258988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-99eb9442-7fe5-4087-87ac-3f5c757c376e req-f3f30ec1-1e6a-4a8e-a203-7a2a104005ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7d10b3f0-fe83-49d1-93c0-013f2e012d2d", "tag": "1040fd9b-dd5d-4f60-9515-4b26b66dbcba"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:33.273859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-99eb9442-7fe5-4087-87ac-3f5c757c376e req-f3f30ec1-1e6a-4a8e-a203-7a2a104005ae service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.274951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-99eb9442-7fe5-4087-87ac-3f5c757c376e req-f3f30ec1-1e6a-4a8e-a203-7a2a104005ae service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.326247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.327903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-99eb9442-7fe5-4087-87ac-3f5c757c376e req-f3f30ec1-1e6a-4a8e-a203-7a2a104005ae service nova] Creating event network-changed:1040fd9b-dd5d-4f60-9515-4b26b66dbcba for instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d on hv-cinder-84902 Jul 20 14:48:33.339485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-99eb9442-7fe5-4087-87ac-3f5c757c376e req-f3f30ec1-1e6a-4a8e-a203-7a2a104005ae service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086217 Jul 20 14:48:33.343712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 40/73] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:33.344433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.346152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.347352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34d82bd3-474b-43b0-bb83-f30a7a558bb5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.514386 Jul 20 14:48:33.352760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:33.353857 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:33.360679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 34/74] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:33.367336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.367951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.368150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:33.368771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:33.369295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.369959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.375405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.375882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.376667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.377214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.513098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.523886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1094f7e-49f8-4623-9c6b-6fd4ba8ba20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1234 microversion: 2.1 time: 0.161599 Jul 20 14:48:33.535776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 35/75] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1234 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:33.541552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:33.542260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:33.549233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.549688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.551633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.551633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.756453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.767891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-176eeddd-034f-4ec9-a2ed-1a58408825bb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.421061 Jul 20 14:48:33.779300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 41/76] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:33 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:33.792379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-08ac4c6d-8a8a-4c20-b737-48cd5b8dd49b req-89b22b48-fa0a-4208-9604-b633f7d6dea5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:33.802126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-08ac4c6d-8a8a-4c20-b737-48cd5b8dd49b req-89b22b48-fa0a-4208-9604-b633f7d6dea5 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:33.802898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-08ac4c6d-8a8a-4c20-b737-48cd5b8dd49b req-89b22b48-fa0a-4208-9604-b633f7d6dea5 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:33.817112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-08ac4c6d-8a8a-4c20-b737-48cd5b8dd49b req-89b22b48-fa0a-4208-9604-b633f7d6dea5 service nova] Creating event network-changed:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:48:33.824308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-08ac4c6d-8a8a-4c20-b737-48cd5b8dd49b req-89b22b48-fa0a-4208-9604-b633f7d6dea5 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042894 Jul 20 14:48:33.825078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 42/77] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:33.868473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:33.878433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-33c61f2a-6c9b-4da1-bcbb-e5c7e49c17fc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.341179 Jul 20 14:48:33.879157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 36/78] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:33 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:34.267733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:34.268470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:34.276020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.276604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.277394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.277924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.366364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:34.367575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:34.374399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.375224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.376034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.376468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.517041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.526091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8dca7d4e-7c88-4eff-aa1b-ca49414b044e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.263197 Jul 20 14:48:34.526725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 43/79] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:34.543477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:34.544356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:34.551141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.551666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.552467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.552905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.565008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.574288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.576122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.577709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f3a62ec0-805c-490d-bb10-3de9c2753eba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.216130 Jul 20 14:48:34.578336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 37/80] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:34.620269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.630523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-22e549b4-2606-4c70-aa8a-2300b77cc31d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1234 microversion: 2.1 time: 0.090975 Jul 20 14:48:34.637560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 44/81] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1234 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:34.785737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:34.786763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:34.793367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.793815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.794576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.795213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.896001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:34.896743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:34.903597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.904014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.916278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:34.917979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:34.958767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:34.967362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27d23d49-0300-4bf3-98e7-3c1facf7a99b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.185885 Jul 20 14:48:34.968560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 38/82] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:34 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:35.073891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:35.086784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b89f3681-d07d-4328-bfdd-f688d79c8481 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.194782 Jul 20 14:48:35.087957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 45/83] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:34 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:35.544266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:35.545001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:35.560988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:35.561891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:35.564634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:35.565284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:35.597743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:35.598367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:35.619630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:35.620089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:35.620889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:35.621317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.002291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.012729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-384b76a1-8af8-4080-8c38-83054b914dc7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.472170 Jul 20 14:48:36.013490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 39/84] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:36.018953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:36.019582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:36.038894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.039993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.041327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.041821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.084046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.093201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.096287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.098200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-04322259-e446-4d4f-9a92-0285a04cd144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.504299 Jul 20 14:48:36.110190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 46/85] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:36.117826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:36.123210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:36.131246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.131865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.134415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.135205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.136495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.146202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-266de19f-4bdb-4dc0-a07c-13721a2570a9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1290 microversion: 2.1 time: 0.131446 Jul 20 14:48:36.146944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 40/86] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1290 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:36.152084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:36.152928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:36.163924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.164854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.166113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:36.166833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:36.380973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.396410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4c8f203e-af88-40b6-956f-cbed633fedbb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.279848 Jul 20 14:48:36.397119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:36.399339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 47/87] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:36 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:36.422081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2238f82d-c995-4172-ae3c-5fa540d26a8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.273253 Jul 20 14:48:36.423455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 41/88] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:36 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:37.030254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:37.030862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:37.037852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.038306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.039100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.039554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.124383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:37.124383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=5 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:37.129098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:37.129716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:37.136339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.136786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.137570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.137997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.308145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.323741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c8aeeb97-2cb5-4854-885f-56fe8b2176c0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.297473 Jul 20 14:48:37.327574 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 48/89] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:37 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:37.334688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:37.335883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:37.344597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.345696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.347020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.347714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.370546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.382000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.383519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.384682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1a198e88-6a32-4f48-8abf-41dc13f5a794 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.271713 Jul 20 14:48:37.385492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 42/90] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:37.417401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:37.418090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:37.432057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.432057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.443482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.444349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.465680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.489991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9367aba0-1d99-4dfd-9e2f-f2c2667a4019 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.160739 Jul 20 14:48:37.490591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 49/91] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:37 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:37.500225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:37.501220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:37.508188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.508931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.509731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:37.510199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:37.755654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.778731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9129fac5-e8bd-4f52-858c-bf55f674a2d9 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.361917 Jul 20 14:48:37.778731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 43/92] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:37 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:37.805885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:37.810544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:37.816500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a2c7fcdd-a558-4cac-993c-3b973d35c92c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.321847 Jul 20 14:48:37.817096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 50/93] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:37 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:38.343399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:38.344086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:38.350886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.351383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.352206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.352748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.406280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:38.409956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:38.417905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.418448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.419443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.420004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.620339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:38.628395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab850571-dc42-4f0a-bbca-3a4f8d7f2809 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.289103 Jul 20 14:48:38.629268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 44/94] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:38 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:38.636449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:38.637433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:38.644422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.644956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.645978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.646571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.652920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:38.662178 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:38.663756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:38.665126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-657358fd-a8e1-4da1-ab9a-6a2e43b516b4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.265645 Jul 20 14:48:38.665698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 51/95] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:38.713370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:38.723261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d306f6a7-9084-4094-96d3-d114cf39d682 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.092661 Jul 20 14:48:38.724043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 45/96] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:38.800758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:38.801469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:38.803591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=6 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:38.810178 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.810732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.811583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.812057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.840884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:38.841731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:38.849067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.849861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:38.850667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:38.851295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.170501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.181830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0ef51797-950b-4411-a4b5-01db5777dd6a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.385103 Jul 20 14:48:39.183152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 52/97] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:38 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:39.195490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.205169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3bed6cb4-b3dd-4d2e-a855-729f4e4e94d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.368673 Jul 20 14:48:39.206639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 46/98] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:38 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:39.649011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:39.649752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:39.659643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.660186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.661030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.661516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.694296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:39.694968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:39.702820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.703642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.704725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.705404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.938600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.957506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2284b54e-c1c7-4f83-abb0-21aa142ce206 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.313785 Jul 20 14:48:39.958744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 53/99] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:39.968420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:39.969318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:39.978083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.985565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.986009 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.986786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:39.987250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:39.993402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.994919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:39.996146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-227773ad-e247-408a-91e9-44800e653391 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.314203 Jul 20 14:48:39.998230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 47/100] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:40.085634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:40.099512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-759ba553-32c1-4cb3-ab29-7d7060fa5b68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.138513 Jul 20 14:48:40.112925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 54/101] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:40.206807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:40.207487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:40.218787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:40.220047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:40.221227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:40.221830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:40.233474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:40.240258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:40.251257 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:40.252415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:40.254016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:40.255026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:40.502571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:40.511861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b7052928-6fb9-4a27-8ea5-614ced33c567 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.309368 Jul 20 14:48:40.512563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 48/102] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:40 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:40.533487 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:40.544907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0deca859-dc80-4f0b-9022-8fc3c7d9b9be tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.322940 Jul 20 14:48:40.546260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 55/103] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:40 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:40.987710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:40.988635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:40.998293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:40.998823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:40.999645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.000078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.020406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:41.021469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:41.037292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.038054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.039810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.040905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.261925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.264834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.271646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2b1e00b9-fef4-4541-b4f2-6b63b65289f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.290212 Jul 20 14:48:41.272458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 49/104] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:40 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:41.282963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.282963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.282963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4f189084-510f-423e-87fd-75c326d8eb0e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.266434 Jul 20 14:48:41.287665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 56/105] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:41.291918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:41.292575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:41.300613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.301147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.302376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.302503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.307294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:41.376453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.385711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f888ff9-6d1f-474e-b60f-ce9b36ccc0bd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.099944 Jul 20 14:48:41.386321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 50/106] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:41.532858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:41.533617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:41.541519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.542471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.543690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.544387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.573379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:41.574039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:41.588788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.589246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.590062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:41.590505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:41.878007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.892588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-93a99f40-b305-4342-af57-74b88b89a18c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.365675 Jul 20 14:48:41.901394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 57/107] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:41 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:41.912553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:41.921799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5fd3c814-4c66-47fe-acbe-d64f1784ddf3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.356279 Jul 20 14:48:41.934649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 51/108] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:41 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:42.310181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:42.310862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:42.311396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:42.312068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:42.326420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.327280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.327910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.330002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.330331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.331446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.332109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.332712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.644002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:42.671353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8cb966e1-fac0-421c-ab20-ae2e86a31bd9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.383928 Jul 20 14:48:42.672777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:42.676949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 58/109] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:42.681916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:42.683872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:42.685356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86d9c439-ea07-48d1-a2e9-a3d63135e0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.394037 Jul 20 14:48:42.685500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:42.685953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 52/110] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:42.686554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:42.697448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.698209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.699583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.700343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.784819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:42.798460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-56c54f43-353e-4f88-b441-5a79370c971c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.120034 Jul 20 14:48:42.799431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 59/111] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:42.913520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:42.914233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:42.924507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.925306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.926992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.927586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.942368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:42.943028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:42.953188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.954022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:42.955156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:42.955828 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:43.196915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:43.203096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:43.210689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3e18e1a7-f7b8-4bab-8290-0fea3d88e631 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.301182 Jul 20 14:48:43.212492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 53/112] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:42 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:43.213667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-97efc315-b8bc-4e25-858b-e5bf8d53673a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.275862 Jul 20 14:48:43.214511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 60/113] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:42 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:43.693640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:43.704615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:43.705007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:43.705390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:43.713135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:43.713618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:43.714504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:43.714967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:43.720438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:43.723697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:43.724533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:43.725854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:43.993381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:44.016739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:44.018362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:44.020356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-86e9973b-dcec-4b3f-a44a-6d238189f266 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.320132 Jul 20 14:48:44.021732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 61/114] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:44.024466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:44.033386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-690a5c9c-8fec-4dad-a784-76534f5da5a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.344795 Jul 20 14:48:44.034230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 54/115] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:44.038334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:44.039005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:44.047550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.048084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:44.048871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.049849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:44.125230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:44.134791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3df33635-1319-4914-aaab-be3d7451e734 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1298 microversion: 2.1 time: 0.110945 Jul 20 14:48:44.135533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 62/116] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1298 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:44.234333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:44.235148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:44.235490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:44.236237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:44.245875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.246477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:44.248613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.255288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:44.256991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.259915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:44.260920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:44.261689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.107606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.116161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1ae205e5-1dcc-497d-b639-76c8a8494644 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.886424 Jul 20 14:48:45.116772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 63/117] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:44 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 888 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:45.135799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.145888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-19ae50e9-1d09-4802-907b-d9ca310743d4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.920050 Jul 20 14:48:45.151280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:45.152170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 55/118] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:44 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:45.154473 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:45.165962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:45.167032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:45.171735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.173966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.175023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.177325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.178136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.181205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.182726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.185556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.694263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.706372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab062121-655d-453a-8493-20b912ff7da6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.550025 Jul 20 14:48:45.707244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 56/119] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:45.718897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:45.719880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:45.742894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.743627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.744605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:45.745202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:45.846087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.864767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.866975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.868522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4d0841dd-9b9c-42f0-9e61-1f2f888a92f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.721435 Jul 20 14:48:45.870143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 64/120] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 753 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:45.910242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:45.924332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fa6f5665-037b-4bb3-b4cf-efa3b8d6b0e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1296 microversion: 2.1 time: 0.214265 Jul 20 14:48:45.925617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 57/121] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1296 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:46.137715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:46.139780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:46.154510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.155848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.157262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.157836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.171322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:46.172854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:46.185055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.185505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.189097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.190866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.447570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:46.459709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:46.469438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9ad15b62-f19d-4e32-81e1-233f2d3a5c33 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.338964 Jul 20 14:48:46.470431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 65/122] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:46 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:46.474813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-173421dc-1b2b-426a-923b-5a42b5b3973f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.309842 Jul 20 14:48:46.475606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 58/123] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:46 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:46.724264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:46.725443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:46.736898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.737461 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.738367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.742982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.887267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:46.887874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:46.896119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.896553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:46.897809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:46.900594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.074481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.088876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-91d2eae9-d7eb-4de5-9c14-5ff2cb5a8e3e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.368482 Jul 20 14:48:47.090529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 66/124] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:46 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:47.105230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:47.105839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:47.122859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.124356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.125335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.125915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.171198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:47.507697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.524377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.527712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.529904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-edfbefd5-4ea2-45c1-a7ff-6b3c99c2bdc2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.646335 Jul 20 14:48:47.531988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 59/125] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:47.538696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:47.539920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:47.554467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.554942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.555954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.558464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.587678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.595521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8af63eee-4af8-4dc9-920a-37053fa2b50c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.503833 Jul 20 14:48:47.619244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 67/126] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:47.625443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:47.626534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:47.638920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.640668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.642276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:47.643411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:47.962327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:47.971087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be5f494d-842e-4df2-920f-53da5a146483 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.437703 Jul 20 14:48:47.972039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 60/127] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:47 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:48.080790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:48.109079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ff1756e2-93e5-4e20-b365-d94ed929a44f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.488026 Jul 20 14:48:48.109546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:48.110422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:48.111180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 68/128] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:47 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:48.124129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:48.124129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:48.129057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:48.129057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:48.556017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:48.557925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:48.576605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:48.580340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:48.582200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:48.586882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:49.510257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:49.532259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e9c4697a-7bae-44a8-8515-23d92290dae7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 1.426902 Jul 20 14:48:49.548847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 61/129] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:48 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 1443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:49.554256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:49.565427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:49.582318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:49.583022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:49.583989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:49.587778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:49.689658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:49.710357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:49.732736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:49.736331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-427a9185-4523-4ae4-93eb-7f50773a9b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 1.190654 Jul 20 14:48:49.759889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 69/130] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 1215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:49.770772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:49.773992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:49.796139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:49.796859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:49.799957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:49.801167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.306745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:50.326274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0fecd5cc-5d30-43ea-a8e3-a09353e03107 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.776059 Jul 20 14:48:50.335706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 62/131] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:50.350054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:50.353131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:50.370647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.374160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.376666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.377924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.716577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:50.725769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9bcfca91-f043-4acf-8982-445198ef94cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.964629 Jul 20 14:48:50.775822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 70/132] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:49 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 1015 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:50.778546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:50.790852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a5b11171-e6dc-461c-b32b-f6efd989da9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.452558 Jul 20 14:48:50.793478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-b3f4517d-21f8-4369-b745-c9ad7b1a550f req-0ad9491e-b066-4825-9d7f-3b8fdae1bcf9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52220445-7cc1-4635-9566-c6fd58b1fc96", "tag": "715e7be7-dd5f-443a-b42e-d143d247ea7d"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:50.802969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 63/133] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:50 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:50.829999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:50.830641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:50.832508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-b3f4517d-21f8-4369-b745-c9ad7b1a550f req-0ad9491e-b066-4825-9d7f-3b8fdae1bcf9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.833735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-b3f4517d-21f8-4369-b745-c9ad7b1a550f req-0ad9491e-b066-4825-9d7f-3b8fdae1bcf9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.851815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-b3f4517d-21f8-4369-b745-c9ad7b1a550f req-0ad9491e-b066-4825-9d7f-3b8fdae1bcf9 service nova] Creating event network-changed:715e7be7-dd5f-443a-b42e-d143d247ea7d for instance 52220445-7cc1-4635-9566-c6fd58b1fc96 on hv-cinder-84902 Jul 20 14:48:50.853939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.857266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.860253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-b3f4517d-21f8-4369-b745-c9ad7b1a550f req-0ad9491e-b066-4825-9d7f-3b8fdae1bcf9 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.081901 Jul 20 14:48:50.861060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 71/134] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:50.863886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.867791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.868701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:50.869610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:50.878900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.879649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:50.880672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:50.881318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.566283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:51.578774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b76612ad-b384-49f7-8a53-cf6034c93b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.763441 Jul 20 14:48:51.579792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 64/135] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:50 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:51.603281 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:51.625797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:51.633833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:51.638055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0b792daa-ae64-4628-8863-571ce8041db6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.772372 Jul 20 14:48:51.639747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 72/136] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:51.646583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:51.648296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:51.657607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.658314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.660886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.661350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.884991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-9ab7b84c-0951-4dd3-b663-abb10344ba89 req-4f6f652a-c22d-439a-a4fa-464ea5d4139c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ad8269f-e328-4587-a622-d5940cd92fa7", "name": "network-vif-plugged", "status": "completed", "tag": "e938812d-42ef-47c6-9b5a-d4cab1fca9f5"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:51.896662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9ab7b84c-0951-4dd3-b663-abb10344ba89 req-4f6f652a-c22d-439a-a4fa-464ea5d4139c service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.897206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9ab7b84c-0951-4dd3-b663-abb10344ba89 req-4f6f652a-c22d-439a-a4fa-464ea5d4139c service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.904053 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:51.913419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b5335523-9f06-4b93-b618-29f773442431 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.272431 Jul 20 14:48:51.927208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 73/137] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:51.929284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-9ab7b84c-0951-4dd3-b663-abb10344ba89 req-4f6f652a-c22d-439a-a4fa-464ea5d4139c service nova] Creating event network-vif-plugged:e938812d-42ef-47c6-9b5a-d4cab1fca9f5 for instance 5ad8269f-e328-4587-a622-d5940cd92fa7 on hv-cinder-84902 Jul 20 14:48:51.935014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-9ab7b84c-0951-4dd3-b663-abb10344ba89 req-4f6f652a-c22d-439a-a4fa-464ea5d4139c service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.349116 Jul 20 14:48:51.935574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 65/138] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:48:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:51.936509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:51.937730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:51.942692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:51.943764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:51.959148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:51.962890 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.963378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.964113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.966084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.966222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.967941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:51.968930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:51.972411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:52.222222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:52.250860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8cffecbb-0a9e-40c6-ade9-83c1456099ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.321734 Jul 20 14:48:52.252772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 74/139] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:51 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:52.350346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:52.368362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d052f369-2494-4541-8cce-ce2c2d12f5fe tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.430963 Jul 20 14:48:52.377395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 66/140] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:51 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:52.596604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:52.597224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:52.607234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:52.607698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:52.608572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:52.609002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:52.656966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:52.657569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:52.671065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:52.671523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:52.672342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:52.672762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:52.714705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:53.151877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.166573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-56721963-efb9-4f7a-9052-213da5218370 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.574065 Jul 20 14:48:53.167579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 75/141] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:52 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:53.184069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:53.186827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:53.197202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.198265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.199354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.199959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.216729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.229873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.234171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.235369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e085f60f-4a53-470a-9a88-945f6ee662b0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.582747 Jul 20 14:48:53.240057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 67/142] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:53.268617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:53.273504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:53.280934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.282142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.283702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.284400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.445138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.458221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-402b0c65-b2c6-4d00-a25b-4900840e0d60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.276810 Jul 20 14:48:53.459595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 76/143] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:53 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:53.465925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:53.485257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.496794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e556ab6-6d4e-42bc-ae3e-de1ca927c6b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.231908 Jul 20 14:48:53.497405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 68/144] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:53 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:53.497521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:53.513812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.514634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.516101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:53.516985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:53.680683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:53.689031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-99116dc4-5512-4d5a-8952-3b2428517097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.227810 Jul 20 14:48:53.689699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 77/145] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:53 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:54.183364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:54.184151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:54.191101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.191667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.192526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.193737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.253507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3af4dea-a30b-44a4-80e3-b51bcafa679e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Action: 'create', calling method: .version_select at 0x7f7b182e6dc0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1854191454", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHkOagv0F7+mg3ne6eKeEEXZacAPENh/gcgDJmtEZoZkQB18S3pHPa+KNQSlD67kqyQn61/hWAGy+RAXiYqunFs536pSeeCiljIz79/rUQKDRWglINqv1UeIF680nKia5Q=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:48:54.269892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-d3af4dea-a30b-44a4-80e3-b51bcafa679e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Getting quotas for project 2c8531e2e07a47eaae4c66d0b82df2d9. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:48:54.274611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-d3af4dea-a30b-44a4-80e3-b51bcafa679e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Getting quotas for user dd610b017e7d4bd786db6bcd08e60722 and project 2c8531e2e07a47eaae4c66d0b82df2d9. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:48:54.305536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3af4dea-a30b-44a4-80e3-b51bcafa679e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.266364 Jul 20 14:48:54.307828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 69/146] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:48:54 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:48:54.327505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:54.328440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:54.348191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.348641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.349427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.350045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.449986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.473020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-79bd29ea-dd2e-4381-af49-a5565a67d3cd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.293293 Jul 20 14:48:54.474742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 78/147] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:54 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:54.497662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:54.498941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:54.507756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:54.511377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.512054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.513020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.513586 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.537364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.547876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.550503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.551815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9660f03a-5eaf-435d-ba1c-35fe8c126787 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.241524 Jul 20 14:48:54.552845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 70/148] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:54.557837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:54.558729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:54.573437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.574208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.575826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.576466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.733896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.742401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1c6782eb-839a-474f-bdfb-0055828bcdee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.266863 Jul 20 14:48:54.743100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 79/149] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:54 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:54.748826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:54.749538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:54.755987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.756464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.757321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:54.757893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:54.789298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.799112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad591d79-4859-4e5c-8160-841d6fb89acc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.244964 Jul 20 14:48:54.799674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 71/150] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:54 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:54.906958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:54.916003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7dd8917c-3017-495b-8b2a-2bbb7992d271 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.171355 Jul 20 14:48:54.916571 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 80/151] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:54 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:55.497057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:55.497654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:55.503630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.504050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.504823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.505224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.571661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:55.572963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:55.580838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.581299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.582149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.582626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.714331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:55.730013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5bb7bfd3-699c-40b4-bbdf-9d35d576293c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.235416 Jul 20 14:48:55.730807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 72/152] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:55.758620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:55.759266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:55.765823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.766266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.767030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.767490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.787488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:55.806375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:55.808844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:55.823282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5dc514dc-fe78-40cd-a8c2-8e17b6f64cab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.243550 Jul 20 14:48:55.825204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 81/153] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:55.838996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:55.840242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:55.852185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.853390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.856909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:55.857873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:55.978637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:55.987452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6e383f7f-2774-49cb-9a2e-6817d4766e8b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.232513 Jul 20 14:48:55.998532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 73/154] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:56.005016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:56.006272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:56.013556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.014232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:56.015177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.015612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:56.306231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:56.320467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:56.329465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-acdb04f2-d853-4ec2-ad0e-26e460ce4e5f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.328099 Jul 20 14:48:56.330539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 74/155] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:55 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:56.333901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1c771955-2af5-49b0-8c33-9a5c433f60a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.506723 Jul 20 14:48:56.344450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 82/156] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:55 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:56.750592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:56.751494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:56.758560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.759311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:56.760520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.760996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:56.849503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:56.851643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:56.859433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.859960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:56.860852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:56.861381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.016243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.024400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16a551ea-6200-4ca7-b6a3-bfba8c5dda52 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.277866 Jul 20 14:48:57.025312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 75/157] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:57.030447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:57.031689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:57.039086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.039778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.040744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.041350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.055945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.071367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.074149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.075450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-96afc6dd-93c7-4371-9d5f-9855cef86810 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.230645 Jul 20 14:48:57.076662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 83/158] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:48:57.213920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.228675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b1a1dd2-5ee0-4781-91e0-a8b21d576aa0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.201828 Jul 20 14:48:57.229453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 76/159] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:57.356511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:57.357430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:57.358557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:57.359462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:57.379994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.380452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.381467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.382002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.382813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.386164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:57.388423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.389116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:57.674474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.683137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:48:57.690216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-88400efb-00b3-4d7c-9083-6aa9dbab7d9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.342072 Jul 20 14:48:57.691062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 84/160] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:57 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:57.711992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:57.732988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0023e98-9df2-4eec-bea5-fb726a901f8b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.380310 Jul 20 14:48:57.733473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 77/161] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:57 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:58.045185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:58.046068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:58.053781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.054274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.055216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.055722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.093921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:58.094806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:58.101466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.101915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.102942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.103655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.274705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.288739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-10e2cd35-0b58-4743-9672-3e080b8e1d68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.250174 Jul 20 14:48:58.298500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 85/162] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:58 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:48:58.298500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:58.298500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:58.308942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.309593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.310456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.311003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.323119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.332683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.334812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.336212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6a15eaa5-aab2-48d7-b7ae-4e7770934fcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.246420 Jul 20 14:48:58.337520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 78/163] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:58.462781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.470743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-26b90d35-d9f9-4bbd-ac01-580c767bd7e8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.179406 Jul 20 14:48:58.471370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 86/164] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:58.710510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:58.711171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:58.719371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.719833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.720697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.721100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.750125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:58.750745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:58.758044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.758579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.759406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:58.760128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:58.941061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.948709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc3bf1d1-0f2b-4054-9b6f-f447d597024e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.243840 Jul 20 14:48:58.949285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 79/165] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:58 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:58.963615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:58.973664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e2d7173e-876c-440f-b7b1-ffd99168b818 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.227500 Jul 20 14:48:58.974260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 87/166] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:58 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:59.307993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:59.309045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:59.316377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.316824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.317599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.318025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.358653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:59.359564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:59.366554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.367021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.367832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.368265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.527817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:59.535900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fee014a1-a708-47e6-b491-f91eddd641d1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.233523 Jul 20 14:48:59.536648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 80/167] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:59.541078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:59.541837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:59.548271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.548713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.549625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.549903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.564878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:59.579114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:59.581872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:59.583825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-79969b02-5d68-43cf-b8d5-86d0cb2d2c77 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.229257 Jul 20 14:48:59.584981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 88/168] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:48:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:48:59.689494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:48:59.701794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c5f252a4-f283-472d-a728-daeea127d827 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.164004 Jul 20 14:48:59.702624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 81/169] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:48:59.966398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:59.967008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:59.973347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.973631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.974460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.974986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.990808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:48:59.991452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:48:59.998221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:48:59.998670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:48:59.999609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.000045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.181484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.192744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2d050315-db54-4121-b1bd-dfbb097062c6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.229997 Jul 20 14:49:00.193342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 89/170] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:59 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:00.198208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.205520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-08a5db0c-0802-4bca-9923-5bd23c367361 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.218269 Jul 20 14:49:00.206421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 82/171] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:48:59 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:00.558357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:00.559611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:00.569776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.570519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.572040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.572763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.604359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:00.604973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:00.612206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.612657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.613580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.614030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.766810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.774608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c2b003f3-e49d-4722-860c-cb2a92b32df4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.222709 Jul 20 14:49:00.775621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 90/172] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:00.780387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:00.781624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:00.788300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.788781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.789514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:00.790107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:00.795336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.804365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.806094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.807452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-057b2f6d-8f58-4c07-8f87-f04df227b398 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.208503 Jul 20 14:49:00.807981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 83/173] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:00.926383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:00.937384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-72b84576-90bc-4eb1-bda6-4475a7f03f0f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.160565 Jul 20 14:49:00.938057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 91/174] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:01.210420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:01.211080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:01.218235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.218723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.219613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.220099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.230295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:01.231030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:01.237633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.238161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.238988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.239548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.416737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:01.427865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e561d230-4db3-4800-bc9e-3920d9e4adf0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.221720 Jul 20 14:49:01.429203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 84/175] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:01 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:01.436155 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:01.444187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-01105ebb-3ea3-4475-9a85-9ce44788a4ea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.219153 Jul 20 14:49:01.444764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 92/176] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:01 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:01.792513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:01.793315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:01.799993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.800431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.801207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.801626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.805976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:01.826611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:01.827220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:01.833639 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.834082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.834906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:01.835395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:01.999721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.008684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a42afaf0-713d-4e9b-b71c-94abea6fbc31 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.220351 Jul 20 14:49:02.009273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 85/177] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:01 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:02.014178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:02.014810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:02.017796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.021912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.022381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.024470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.025113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.026628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.028578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.030166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9105d649-0066-4e45-a75d-5a3460f81109 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.208103 Jul 20 14:49:02.030868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 93/178] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:02.184115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.191896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff191366-bcc3-43b4-9cd1-2be9e047f108 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.181468 Jul 20 14:49:02.192495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 86/179] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:02.445659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:02.446829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:02.453951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.454414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.455266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.455695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.461657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:02.462302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:02.468788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.469235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.469994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:02.470442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:02.675865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.681912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:02.684035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dae96b47-0413-4fae-9729-47f7a04f7e6c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.242372 Jul 20 14:49:02.684986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 94/180] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:02 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:02.691545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1de5b75c-09da-4920-8ce4-edb7f24e309d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.233652 Jul 20 14:49:02.701338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 87/181] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:02 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:03.025855 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:03.026485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:03.033201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.033648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.034406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.034851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.052528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:03.053593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:03.059449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.059889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.060643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.061069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.258627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.268202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-692fccd5-2f2d-4fa7-849b-ddb0358e123c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.246148 Jul 20 14:49:03.271620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.274019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 95/182] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:03.279099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:03.280004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:03.286808 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.287298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.288061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.288486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.293530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.296917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.299266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ac59c85d-3dab-4cf0-9eab-1a040ccb9087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.250327 Jul 20 14:49:03.300504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 88/183] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:49:03.450192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.462228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f760d7d7-9d52-40d4-b903-62d06e3d20ea tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.187109 Jul 20 14:49:03.462845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 96/184] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:03.701437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:03.702062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:03.709854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:03.710481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:03.710605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.710943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.711735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.712164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.717227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.717642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.718593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:03.719568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:03.923295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.932807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:03.938942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-619ec042-9564-44f5-8b53-73442c59b971 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.240358 Jul 20 14:49:03.940328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 89/185] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:03 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:03.945342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd668d44-9c35-4adb-80b6-cde5166d325b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.239441 Jul 20 14:49:03.946422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 97/186] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:03 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:04.291475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:04.292083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:04.294189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:04.301784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.302302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.303243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.303674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.326498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:04.327730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:04.334542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.335232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.336274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.336920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.532570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:04.541093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-998a1e9c-f98f-4d9d-9ad3-3fa5d488b37d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.253869 Jul 20 14:49:04.541698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 90/187] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:04.551443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:04.552069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:04.554223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:04.559784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.560828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.562084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.562736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.563103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:04.564900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:04.566162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-284edc4b-1fbc-4374-b8ed-bcedbd91b387 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.244020 Jul 20 14:49:04.566739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 98/188] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:49:04.715596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:04.724592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f8dd437d-8865-49b4-9ebd-e5f3c1b2bb68 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.181470 Jul 20 14:49:04.725350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 91/189] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:04.956605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:04.957067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:04.963591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:04.964106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.964540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:04.964661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.965293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.965924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.972102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.972728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:04.973631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:04.975321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.189945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.199491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1e551848-18a9-4a53-8d7e-b1811012b116 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.246648 Jul 20 14:49:05.200279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 99/190] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:04 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:05.202778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.216848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4644802-2fab-426b-a39a-a5f857b180a7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.257977 Jul 20 14:49:05.217450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 92/191] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:04 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:05.560399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:05.561086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:05.569253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.569253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.569900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.570371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.585254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:05.586154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:05.593063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.593540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.594502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.594831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.814441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.823915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-09ea5d67-cf80-4524-9012-087c31d58f8b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.268772 Jul 20 14:49:05.824611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 100/192] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:05.835908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.843291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:05.844338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:05.849945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.851523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:05.851931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.852585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.852770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cc904961-6f94-4d0b-a7d7-e90906d5af99 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.271719 Jul 20 14:49:05.853794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:05.853884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 93/193] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:49:05.854463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:05.874456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:06.002311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:06.011936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd3fb4a4-a865-42b9-addb-58cb4d32d239 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.173891 Jul 20 14:49:06.012642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 101/194] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:06.216370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:06.217202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:06.224039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.224491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.227419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.227419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.234272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:06.234968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:06.243481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.244419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.245534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.246193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.428772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:06.436882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a928270a-7d96-4267-864d-a3050d9df49a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.224332 Jul 20 14:49:06.446099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 94/195] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:06 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:06.447927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:06.459591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4e398960-f6f2-43aa-8ff3-3183853833b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.229274 Jul 20 14:49:06.469543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 102/196] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:06 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:06.842990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:06.843758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:06.850604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.851065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.851844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.852271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.874079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:06.875007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:06.883151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.883151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:06.886109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:06.886109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.056267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.064127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb86522c-e63c-41e9-affa-b6405e5704c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.225224 Jul 20 14:49:07.064685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 95/197] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:07.069277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:07.069868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:07.075953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.076396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.077149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.077569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.082761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.091158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.092749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.093960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-760d0d2d-b914-4dd0-ba3b-4be86a3d70dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.225523 Jul 20 14:49:07.094795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 103/198] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:07.223885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.232004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-caaf1bf4-e0a8-49c8-9103-0cf2d131cfe6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.166125 Jul 20 14:49:07.232579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 96/199] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:07 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:07.456988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:07.457532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:07.464397 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.464848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.465676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.466110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.477728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:07.478619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:07.485387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.486486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.486759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:07.487409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:07.676291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.684901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-48405b28-5842-4aef-8608-9792cb18ec5d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.233672 Jul 20 14:49:07.685068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:07.685724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 104/200] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:07 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:07.693356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b150249c-059e-4667-9675-9347eb54307a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.219818 Jul 20 14:49:07.694045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 97/201] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:07 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:08.082608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:08.083247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:08.090718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.091321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.092463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.093041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.121253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:08.121885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:08.128855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.129452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.130417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.130973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.541610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.549333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.550159 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c3a48226-ee2a-4523-9685-2a462518f025 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.471466 Jul 20 14:49:08.550761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 105/202] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:08 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:08.555500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:08.556116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:08.562522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.562845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.562981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.563758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.564184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.564430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.565583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ed9edc29-3fe5-463d-ba3d-c38b4b1f6374 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.457020 Jul 20 14:49:08.566355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 98/203] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:08 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:08.702667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:08.703245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:08.710869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.711344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.712129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.712567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.725319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.737530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-20ea36d1-388c-44b9-906f-3e1020dde88b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.185613 Jul 20 14:49:08.738244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 106/204] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:08.743075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:08.743727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:08.751331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.751849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.752726 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:08.753253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:08.925798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.934639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-70a81ae8-4e4b-4450-a383-a7cd0397689d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.236342 Jul 20 14:49:08.935621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 99/205] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:08 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:08.948662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:08.956533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-776db349-e601-4fca-a0ba-0f32f04358e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.217213 Jul 20 14:49:08.957004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 107/206] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:08 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:09.499383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-ff70d7a5-b8a6-48d8-9f5d-63d4ecd56e0a req-1985af48-6334-47b9-8bc9-a44905cc0377 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "name": "network-vif-plugged", "status": "completed", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:09.509790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-ff70d7a5-b8a6-48d8-9f5d-63d4ecd56e0a req-1985af48-6334-47b9-8bc9-a44905cc0377 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.510237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-ff70d7a5-b8a6-48d8-9f5d-63d4ecd56e0a req-1985af48-6334-47b9-8bc9-a44905cc0377 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.524287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-ff70d7a5-b8a6-48d8-9f5d-63d4ecd56e0a req-1985af48-6334-47b9-8bc9-a44905cc0377 service nova] Creating event network-vif-plugged:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:49:09.530019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-ff70d7a5-b8a6-48d8-9f5d-63d4ecd56e0a req-1985af48-6334-47b9-8bc9-a44905cc0377 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034430 Jul 20 14:49:09.530747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 100/207] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:09.568430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:09.569210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:09.586498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.587332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.588111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:09.588446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.588818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:09.588970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.600457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.601162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.603223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.603667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.823068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:09.832621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:09.834290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:09.835261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:09.835434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-958245ad-26e2-4111-95db-0e9fc4fc8c05 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.254818 Jul 20 14:49:09.836031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 101/208] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:09.842185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:09.843034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:09.844343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9c293ac1-224d-4286-9cad-a1b6cc715bb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.280087 Jul 20 14:49:09.845778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 108/209] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:09 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:09.850446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.851106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.852276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.852957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.873820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:09.954524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:09.955233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:09.961580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.962286 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:09.963153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:09.963582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.020526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:10.028942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0195b692-4edb-4e82-ab66-872f959162b0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.191566 Jul 20 14:49:10.029716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 102/210] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:10.034377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:10.035650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:10.042027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.042472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.043272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.043717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.148301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:10.172141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ea446f11-1dbc-417a-b01c-e6a3993d8abb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.223747 Jul 20 14:49:10.173175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 109/211] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:09 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:10.241265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:10.253748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b369fe4d-595c-42ac-9d85-2a5a9cbe2b72 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.223273 Jul 20 14:49:10.264231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 103/212] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:10 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:10.856667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:10.856667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:10.863635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:10.864405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:10.873468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.875432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.876026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.876593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.877640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.878659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:10.879286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:10.879817 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.113398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.116671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.122839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d6c7be4b-6dab-4f93-9a4b-5a134a57d337 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.263789 Jul 20 14:49:11.123959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 104/213] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:10 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:11.126558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.128447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.129444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:11.130119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c269f92a-6b62-498f-ae24-8bc373f0d63f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.280233 Jul 20 14:49:11.130240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:11.132263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 110/214] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:11.137416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.138125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.139036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.139744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.197029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:11.197842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:11.204871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.205770 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.206964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.207694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.366184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.374236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d700df97-d339-4915-a940-2785409c6ae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.248900 Jul 20 14:49:11.385130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 105/215] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:11.392933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:11.393688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:11.400353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.400958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.401903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.402495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.418878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.436618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ce14c9e1-3913-43c6-8879-15f90e0545d6 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.246692 Jul 20 14:49:11.437203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 111/216] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:11 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:11.548717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7d10b3f0-fe83-49d1-93c0-013f2e012d2d", "name": "network-vif-plugged", "status": "completed", "tag": "1040fd9b-dd5d-4f60-9515-4b26b66dbcba"}, {"server_uuid": "304148dc-eb2a-4dec-a9ec-1cff5c5114b7", "name": "network-vif-plugged", "status": "completed", "tag": "c215e006-b33e-46f8-be1f-7cc86ace4f37"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:11.557975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:11.558452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:11.562158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:11.570072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7d8c6c8c-7c2f-449b-ae45-100820a622eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.183513 Jul 20 14:49:11.570653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 106/217] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:11 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:11.573320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] Creating event network-vif-plugged:1040fd9b-dd5d-4f60-9515-4b26b66dbcba for instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d on hv-cinder-84902 Jul 20 14:49:11.573913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] Creating event network-vif-plugged:c215e006-b33e-46f8-be1f-7cc86ace4f37 for instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 on hv-cinder-84902 Jul 20 14:49:11.580637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-77ed48b1-c552-464b-8255-d74bade87be4 req-c6bd1ae0-0728-4db1-b620-e0a842efdd76 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.035680 Jul 20 14:49:11.581229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 112/218] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:11 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:49:12.149197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:12.149992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:12.150915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:12.151791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:12.157921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.158381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.159353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.159771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.160134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.160582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.161382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.161820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.372417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.381654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.384148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-614488c2-3864-41a1-a98f-e90af782849d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.237280 Jul 20 14:49:12.384822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 113/219] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:12 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:12.391342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.391790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:12.392462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:12.393268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.394428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f81899f7-f780-4291-89fe-993f8c8fb108 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.249526 Jul 20 14:49:12.394957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 107/220] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:12.399502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.399889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.400694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.401160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.454670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:12.456450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:12.464327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.464771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.465550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.466037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.587845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.596218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5b0a91f2-c3fa-432a-b6ae-e44d52e229b9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.208454 Jul 20 14:49:12.597071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 114/221] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:12.602608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:12.603271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:12.612158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.612873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.614002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:12.614652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:12.649277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.663933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8acc50d3-1001-4d41-a090-82e9597fe9fd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1314 microversion: 2.1 time: 0.213799 Jul 20 14:49:12.673420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 108/222] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:12 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1314 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:12.781887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:12.790162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4b56d665-aa34-49f4-9e01-8f1b5482b466 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.191460 Jul 20 14:49:12.790839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 115/223] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:12 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:13.401641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:13.402847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:13.410827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.411306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.412063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.412524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.412672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:13.413280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:13.422389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.422890 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.423766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.424203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.639039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.646511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.655111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.655628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f022f96e-c7b4-4c29-a112-16c8dfa2aa97 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.257920 Jul 20 14:49:13.656661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 109/224] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:13.656910 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.658249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c75f4e9e-076a-4b02-b8db-752c308382db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1407 microversion: 2.20 time: 0.249487 Jul 20 14:49:13.658926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 116/225] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1407 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:13.662756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:13.662756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:13.670768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.671289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.672087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.672515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.682595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:13.683526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:13.692699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.693596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.694827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.695424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.872794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.881249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a944c898-7f24-478e-ba8b-83a1bcf22dba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.223302 Jul 20 14:49:13.881907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 110/226] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:13.886811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:13.887450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:13.890301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:13.892634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:13.898201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.898714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.899805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:13.900331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:13.903592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-138eb3fe-0a2b-44a3-992b-1d2d3faa66ec tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1501 microversion: 2.1 time: 0.225996 Jul 20 14:49:13.904176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 117/227] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:13 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1501 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:14.153507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:14.164491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f0068d53-df85-44bd-9951-91aa50b609a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.280747 Jul 20 14:49:14.165772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 111/228] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:13 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:14.674038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:14.674651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:14.681213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.681575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.682347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.682775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.684943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:14.685804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:14.692991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.693644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.695005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.695851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.890138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:14.894481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:14.899028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:14.900417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ac7f51b7-89d8-49b4-b657-04576ed593e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.230353 Jul 20 14:49:14.901325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 118/229] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:14.906562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:14.907249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:14.908102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:14.909596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:14.910755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9a2c3fe0-74bb-4439-9a88-fac298dd944d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1600 microversion: 2.20 time: 0.232245 Jul 20 14:49:14.911338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 112/230] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:49:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1600 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:49:14.914244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.914690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.915521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.915949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.929683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:14.930179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:14.936650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.937023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:14.937928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:14.938363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:15.080510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-691a3620-357b-45f2-8a05-80343fe98fbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.154962 Jul 20 14:49:15.081179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 113/231] 10.222.0.15 () {60 vars in 1315 bytes} [Wed Jul 20 14:49:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-interface => generated 285 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:49:15.129473 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:15.137423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d163420-a69b-4823-99ad-4df9b7fee42d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.234852 Jul 20 14:49:15.138052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 119/232] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:15.181909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:15.182519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:15.189212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:15.189590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:15.190360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:15.190946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:15.491357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:15.498997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d0287da3-8fb8-4c52-936a-244885295488 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.321034 Jul 20 14:49:15.499612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 114/233] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:15 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:15.918254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:15.918839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:15.925488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:15.925932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:15.926707 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:15.927187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.156815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:16.157600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:16.164104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:16.164799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.165594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:16.166086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.228910 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:16.237711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c8ffb5da-9c2b-4c6e-901d-8c7f74f6828d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.323241 Jul 20 14:49:16.238300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 120/234] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:16.408311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:16.416547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7b246684-3c82-4966-84a1-b533355cdb3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.265089 Jul 20 14:49:16.417140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 115/235] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:16.516495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:16.517290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:16.523922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:16.524521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.525307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:16.525849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.742222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:16.750676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-760c6127-f3d0-495c-abfe-70357637d45b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.238121 Jul 20 14:49:16.751371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 121/236] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:16 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:16.816513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-63c0795c-338b-48b2-8e73-85c69e5717ca req-8815100b-51ec-4de3-a658-94f99f7ef36f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:16.825772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-63c0795c-338b-48b2-8e73-85c69e5717ca req-8815100b-51ec-4de3-a658-94f99f7ef36f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:16.826097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-63c0795c-338b-48b2-8e73-85c69e5717ca req-8815100b-51ec-4de3-a658-94f99f7ef36f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:16.840533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-63c0795c-338b-48b2-8e73-85c69e5717ca req-8815100b-51ec-4de3-a658-94f99f7ef36f service nova] Creating event network-changed:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:49:16.846364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-63c0795c-338b-48b2-8e73-85c69e5717ca req-8815100b-51ec-4de3-a658-94f99f7ef36f service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034892 Jul 20 14:49:16.848097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 116/237] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:17.254435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:17.255199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:17.262236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.262781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.263595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.264033 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.409552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:17.419943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-874315f0-5151-41ef-87df-a39888583574 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.169626 Jul 20 14:49:17.420533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 122/238] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:17 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:17.434741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:17.435593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:17.442204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.442639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.443426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.443857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.586146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:17.594007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c998955-f0f5-4cdc-889f-1af9cf530c4e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.163667 Jul 20 14:49:17.594842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 117/239] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:17.772934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:17.773545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:17.780201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.780975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.781756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:17.782187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:17.930032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:17.938516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f04838fa-eaed-4533-8613-c690c358d36e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.170723 Jul 20 14:49:17.940319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 123/240] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:17 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:18.437476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:18.438146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:18.444812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.445240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:18.446236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.446739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:18.595785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:18.603414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-741516b1-32b9-4f26-a2fb-87ed7bd2eb6c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.169932 Jul 20 14:49:18.603978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 118/241] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:18 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:18.612306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:18.612914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:18.619060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.619519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:18.621472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.621472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:18.757654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:18.766227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3f2653f0-6002-4ceb-b69e-54f89c65476d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.157874 Jul 20 14:49:18.766787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 124/242] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:18.956812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:18.957446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:18.964553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.965004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:18.965778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:18.966213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:19.100766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:19.108504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d073e060-ef05-4ba7-abcf-35d39c08e43f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.155782 Jul 20 14:49:19.109080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 119/243] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:18 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:19.620869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:19.621490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:19.628074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:19.628511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:19.629265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:19.629687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:19.772742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:19.785815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-caf015ff-684e-4748-a3c5-d91840d3d8e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.168832 Jul 20 14:49:19.786482 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 125/244] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:19.788848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:19.789489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:19.796299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:19.796744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:19.797519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:19.798047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:19.802868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:19.943376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:19.951422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d0fb8f0-4b97-4842-8403-35ae94ea3290 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.166478 Jul 20 14:49:19.952005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 120/245] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:19 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:20.126083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:20.126543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:20.133564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.134064 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:20.134981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.135701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:20.272835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:20.282018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-153ca0e6-101f-4660-b759-9c161366aaec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.160066 Jul 20 14:49:20.282652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 126/246] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:20 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:20.808507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:20.809175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:20.815821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.816268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:20.817062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.817492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:20.958235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:20.970093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:20.970804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2152cd6e-f9e5-49ca-8da8-3e98fa35eefa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.165766 Jul 20 14:49:20.971920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 121/247] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:20 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:20.972745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:20.980098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.980550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:20.981302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:20.981924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.082155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-80df2384-60b4-4095-83ba-14cae97fb277 req-9e6dda0f-b729-40bf-9d02-7dec0ba1dc59 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b566c6c-a418-4384-b99e-c7295e754e92", "name": "network-vif-plugged", "status": "completed", "tag": "67324695-43bc-444a-9a35-3a3bbb87de0b"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:21.095207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-80df2384-60b4-4095-83ba-14cae97fb277 req-9e6dda0f-b729-40bf-9d02-7dec0ba1dc59 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.096042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-80df2384-60b4-4095-83ba-14cae97fb277 req-9e6dda0f-b729-40bf-9d02-7dec0ba1dc59 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.108324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-80df2384-60b4-4095-83ba-14cae97fb277 req-9e6dda0f-b729-40bf-9d02-7dec0ba1dc59 service nova] Creating event network-vif-plugged:67324695-43bc-444a-9a35-3a3bbb87de0b for instance 6b566c6c-a418-4384-b99e-c7295e754e92 on hv-cinder-84902 Jul 20 14:49:21.114442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-80df2384-60b4-4095-83ba-14cae97fb277 req-9e6dda0f-b729-40bf-9d02-7dec0ba1dc59 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036550 Jul 20 14:49:21.115070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 122/248] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:49:21.122732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:21.131016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7786bb4b-cf3e-4074-9127-1c38a2c5549d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.165574 Jul 20 14:49:21.131757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 127/249] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:21.302414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:21.303146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:21.311292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.311739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.312537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.312965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.460063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:21.479868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab6129c4-73b3-492a-9a93-7f6ed1714c60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.182968 Jul 20 14:49:21.480700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 123/250] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:21 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:21.876415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "42d53966-5482-4f32-8c1e-8dbf89162071"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:21.878156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:21.888762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.890340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.891156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.891630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.988825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:21.989446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:21.996076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.996549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:21.997321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:21.997751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:22.142107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:22.149912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e1d0f0ce-2bc7-4b1c-9353-1ce5a8c46008 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.165030 Jul 20 14:49:22.150488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 124/251] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:22.155028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:22.155994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:22.162408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:22.162935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:22.163793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:22.164278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:22.299077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:22.307782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2988ce5c-fe8c-4c3f-abbd-e50be2b93bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.156279 Jul 20 14:49:22.308371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 125/252] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:22.501958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:22.502627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:22.509526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:22.510005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:22.510904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:22.511444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:22.652011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:22.660397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-42476340-8e50-491a-b306-dc13c5198097 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.164721 Jul 20 14:49:22.660983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 126/253] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:22 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:23.168318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:23.169046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:23.175804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.176469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.177498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.178265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.318873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:23.326834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44a83c03-acec-4272-b8ba-b6f0d2e205bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.162566 Jul 20 14:49:23.327455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 127/254] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:23.331919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:23.332521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:23.339630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.340361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.341567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.342160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.486472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:23.494631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5a347ffd-c1e8-4fc5-b78a-d531a7dca2fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.166183 Jul 20 14:49:23.495238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 128/255] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:23.677878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:23.678415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:23.685473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.685910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.686772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:23.687224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:23.839215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:23.847833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dc56befc-69a4-4fb6-a315-785c9de07bf9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.174134 Jul 20 14:49:23.848423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 129/256] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:23 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:24.344919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:24.345936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:24.356883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.357339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:24.358110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.358543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:24.532685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:24.540733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-38c5b5fe-231a-485f-a98e-db0cc29e72b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.200833 Jul 20 14:49:24.541377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 130/257] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:24.546053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:24.546659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:24.552977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.553420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:24.554188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.554615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:24.701568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:24.710813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b5b3dfb-5d5a-4694-a338-d54516087a78 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.168312 Jul 20 14:49:24.711417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 131/258] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:24.864763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:24.865368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:24.872108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.872958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:24.873739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:24.874176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:25.015481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:25.023144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f519367-e633-4b73-ad75-cb38b0b6f798 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.162147 Jul 20 14:49:25.023723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 132/259] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:24 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:25.129344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1693245306", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1854191454", "security_groups": [{"name": "tempest-secgroup-smoke-1887982402"}], "block_device_mapping_v2": [{"uuid": "5c2ef894-04fd-4740-8fe5-d875160edaf9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "64e0be66-24a7-4de2-8e70-9a2ac078a072"}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:25.259146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] validate_networks() for [('64e0be66-24a7-4de2-8e70-9a2ac078a072', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:49:25.692130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:49:25.692719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:49:25.692993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:49:25.693392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:49:25.693791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:49:25.705561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:25.706178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:25.706859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:25.707299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:25.726576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Getting quotas for project 2c8531e2e07a47eaae4c66d0b82df2d9. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:49:25.731306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Getting quotas for user dd610b017e7d4bd786db6bcd08e60722 and project 2c8531e2e07a47eaae4c66d0b82df2d9. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:49:25.791640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:49:25.793797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='5c2ef894-04fd-4740-8fe5-d875160edaf9',volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:49:25.972348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4fcf1800-0ff5-4174-857e-94f0fc4db61a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.846818 Jul 20 14:49:25.972997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 133/260] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:49:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 848 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:49:25.981659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-d665ba5d-5e10-4f34-8ed6-ca4d806d9535 req-95ef2bc2-52ae-4bcc-90c1-a820cbce884d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52220445-7cc1-4635-9566-c6fd58b1fc96", "name": "network-vif-plugged", "status": "completed", "tag": "715e7be7-dd5f-443a-b42e-d143d247ea7d"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:25.990605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d665ba5d-5e10-4f34-8ed6-ca4d806d9535 req-95ef2bc2-52ae-4bcc-90c1-a820cbce884d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:25.991157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d665ba5d-5e10-4f34-8ed6-ca4d806d9535 req-95ef2bc2-52ae-4bcc-90c1-a820cbce884d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.003168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-d665ba5d-5e10-4f34-8ed6-ca4d806d9535 req-95ef2bc2-52ae-4bcc-90c1-a820cbce884d service nova] Creating event network-vif-plugged:715e7be7-dd5f-443a-b42e-d143d247ea7d for instance 52220445-7cc1-4635-9566-c6fd58b1fc96 on hv-cinder-84902 Jul 20 14:49:26.008295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-d665ba5d-5e10-4f34-8ed6-ca4d806d9535 req-95ef2bc2-52ae-4bcc-90c1-a820cbce884d service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030386 Jul 20 14:49:26.008988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 134/261] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:26.017231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:26.017921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:26.024348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.024793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.025553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.025970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.197823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:26.206147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc1ec84a-2fee-42fc-8404-5c7fbe11717e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.193037 Jul 20 14:49:26.215159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 135/262] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:26.215159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:26.215159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:26.221774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.222217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.223004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.223459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.367992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:26.376147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c4d4ac87-6db3-4a7f-be68-ba4de92d938f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.167337 Jul 20 14:49:26.376814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 136/263] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:26.381412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-982728e5-4660-401b-a29f-4de6d82fe72d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:26.382226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-982728e5-4660-401b-a29f-4de6d82fe72d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:26.443192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-982728e5-4660-401b-a29f-4de6d82fe72d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:26.450368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-982728e5-4660-401b-a29f-4de6d82fe72d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1095 microversion: 2.1 time: 0.072513 Jul 20 14:49:26.450945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 137/264] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:26 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1095 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:26.455640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:26.456235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:26.462533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.462969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.463765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:26.464190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:26.603661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:26.614727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3969dcd5-61fd-4ee0-aca1-ad4ead5fa25c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1322 microversion: 2.1 time: 0.162429 Jul 20 14:49:26.615379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 138/265] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:26 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1322 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:27.223923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:27.224775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:27.242095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.243021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.244013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.244665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.430145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:27.434887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:27.439712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b63b55ac-4925-4ddf-8158-bd495a04ce3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.220320 Jul 20 14:49:27.440940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 139/266] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:27 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:27.446269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:27.446870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:27.453541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.454273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.455083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.455507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.468309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a768fa7a-f86e-43db-bf7c-ae9b3df1c95a tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "POST /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.596386 Jul 20 14:49:27.474842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 128/267] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:49:21 2022] POST /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7/os-volume_attachments => generated 194 bytes in 5600 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:27.482631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:27.485648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:27.488347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=99 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:27.495095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.495285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.496062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.496515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.605559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:27.613372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-da280325-78d9-4051-926b-4527149e9d61 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1145 microversion: 2.1 time: 0.142176 Jul 20 14:49:27.614187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 129/268] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:27 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1145 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:27.640020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:27.640891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:27.648654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.649568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.649789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.650231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.690582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:27.701178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-83b719df-dbc5-4493-a44c-1283e015ac12 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.258728 Jul 20 14:49:27.718218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 140/269] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:27 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:27.861485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:27.869850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-301efa74-64b4-495b-a988-b773de900322 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1515 microversion: 2.1 time: 0.235550 Jul 20 14:49:27.870445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 130/270] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:27 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1515 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:27.891887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:27.894473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:27.902367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.904101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.905002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:27.906019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:27.932902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'flavor' on Instance uuid 7d10b3f0-fe83-49d1-93c0-013f2e012d2d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:49:28.090325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45b466f6-cca1-40ab-8d45-7bccc4756f37 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d/action" status: 200 len: 32 microversion: 2.1 time: 0.206662 Jul 20 14:49:28.091165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 141/271] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:49:27 2022] POST /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d/action => generated 32 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 14:49:28.100070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:28.100950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:28.107412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.107864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.108643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.109096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.262217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:28.270537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cc081bd2-66c0-4cb0-8b5a-459fcc7baebe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.174253 Jul 20 14:49:28.271129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 131/272] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:28 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:28.458318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:28.459766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:28.469510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.469959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.470722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.471168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.630656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:28.631291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:28.632227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:28.639568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d99dc62-5433-469f-8615-95029e557eb7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.186019 Jul 20 14:49:28.639867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.640318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.641560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.642198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.649782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 142/273] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:28.730071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:28.734394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:28.735665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:28.742742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.742976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dfef52ec-4538-41bb-8963-4a886046f7bd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1193 microversion: 2.1 time: 0.116414 Jul 20 14:49:28.743549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 132/274] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:28 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1193 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:28.744352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.745486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:28.746142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:28.894238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:28.902247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4d5fb8c-3089-4254-a117-07823af897ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.171513 Jul 20 14:49:28.902866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 143/275] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:29.288211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:29.288871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:29.297337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.298263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.299503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.300222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.459962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:29.468334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-998aa160-0296-4942-8b56-49dd990ab659 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.184105 Jul 20 14:49:29.468964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 133/276] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:29 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:29.658957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:29.660519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:29.668233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.668810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.669701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.670199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.763823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:29.764442 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:29.771540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.772219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.772885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.773328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.777702 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:29.879495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:29.886659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:29.891451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ddd1bafa-4976-4852-bc1a-bbb05a3f0644 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1193 microversion: 2.1 time: 0.132270 Jul 20 14:49:29.894003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 134/277] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:29 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1193 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:29.905989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e6dc484a-b412-4847-ac0a-f526ad48ff7d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.251261 Jul 20 14:49:29.907233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 144/278] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:29.927768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:29.929208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:29.937796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.938326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:29.939622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:29.940131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.127379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:30.137919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c44fcbbc-ae78-4fa3-b35a-c03f33a88d0a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.219457 Jul 20 14:49:30.137919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 135/279] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:30.486151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:30.486775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:30.495460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.496055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.496874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.497365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.674002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:30.683194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4533f658-fa5f-4016-9653-c076778f2767 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.201004 Jul 20 14:49:30.683977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 145/280] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:30 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:30.910898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:30.911570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:30.928764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:30.929036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.929348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:30.929490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.930169 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.930864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.938186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.938978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:30.941203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:30.941616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:31.052110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:31.067326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9225d475-d523-42c4-b601-adbccf778160 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1193 microversion: 2.1 time: 0.160161 Jul 20 14:49:31.078228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 136/281] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:30 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1193 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:31.154049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:31.154962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:31.168385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:31.178097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:31.178097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:31.178097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:31.194703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:31.205928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4bea20ec-ea9d-456c-b71d-a293808a4392 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1366 microversion: 2.1 time: 0.281766 Jul 20 14:49:31.207082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 146/282] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:30 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1366 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:31.348607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:31.356736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ae3edc3c-0013-4138-9caa-e93d7208b9eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.206465 Jul 20 14:49:31.358452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 137/283] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:31.707595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:31.709016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:31.715987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:31.716694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:31.717480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:31.717891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:31.912212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:31.920900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eeb24b7d-592f-40bd-b852-0102da4f02d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.218757 Jul 20 14:49:31.921573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 147/284] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:31 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:32.084837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:32.086179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:32.092798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.093415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.094162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.094907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.169304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:32.179985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0a4c3444-fe53-4055-aba6-20b0d9a3f76e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1201 microversion: 2.1 time: 0.098764 Jul 20 14:49:32.180805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 138/285] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:32 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1201 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:32.235287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:32.235897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:32.242758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.243581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.244590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.245006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.375280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:32.376255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:32.384181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.384618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.385576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.386106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.688143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:32.698471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c9386cfb-20e7-480d-a967-73d25bb6e358 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1363 microversion: 2.1 time: 0.327662 Jul 20 14:49:32.702250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 139/286] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1363 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:32.747564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:32.764218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5ff5365e-56c4-4039-9c9c-b63fbff1de4c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1555 microversion: 2.1 time: 0.533187 Jul 20 14:49:32.765517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 148/287] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1555 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:32.784808 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:32.785584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:32.793146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.794140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.795244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.795695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.912027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5267b8e3-16a0-4cd6-af24-9456682302a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.132009 Jul 20 14:49:32.912880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 140/288] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 14:49:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-interface => generated 285 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:32.942682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:32.943777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:32.951760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.952806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:32.954203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:32.954648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.127550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:33.141040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-70855971-bda3-46cd-ab13-83985e33de38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.206366 Jul 20 14:49:33.142277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 149/289] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:32 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:33.198231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:33.199200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:33.211393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.211875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.212687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.213157 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.294648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:33.304434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ee221a50-ce4b-4a53-9fc1-29b65a4c3eae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1211 microversion: 2.1 time: 0.111401 Jul 20 14:49:33.315074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 141/290] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:33 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1211 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:33.727436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:33.729493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:33.739473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.740438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.741466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.742111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.903454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:33.912566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86886eb0-982f-4618-9b4e-a6b4abe18b61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1550 microversion: 2.1 time: 0.197440 Jul 20 14:49:33.913241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 150/291] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1550 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:33.930971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:33.931502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:33.938436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.938942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:33.939753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:33.940185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:34.042950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-960c50a6-a83e-4e74-b022-c65a8b20f161 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.116326 Jul 20 14:49:34.043660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 142/292] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 14:49:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-interface => generated 285 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:34.158746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:34.159613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:34.168371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:34.169241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:34.170303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:34.170959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:34.321580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:34.322342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:34.329896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:34.330336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:34.331129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:34.331556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:34.457807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:34.470678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:34.474466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-27b540ce-3621-42c5-af99-06531bba7152 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1211 microversion: 2.1 time: 0.157059 Jul 20 14:49:34.483907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 143/293] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:34 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1211 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:34.488610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34b68ccf-30f7-4c2e-ae7b-87253a455d1d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.334567 Jul 20 14:49:34.503782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 151/294] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:34 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:35.396336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-f83b3bb2-5bbd-4fa9-be77-a62d7c2f855e req-d84f5613-7b0d-4a0f-a5a5-558bf43b5fad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6b566c6c-a418-4384-b99e-c7295e754e92", "tag": "67324695-43bc-444a-9a35-3a3bbb87de0b"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:35.438806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-f83b3bb2-5bbd-4fa9-be77-a62d7c2f855e req-d84f5613-7b0d-4a0f-a5a5-558bf43b5fad service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.442592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-4b94495c-2d5a-4873-a70b-9566260ceb70 req-e522af3d-4d7d-490d-a1d0-e96a4aa380e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52220445-7cc1-4635-9566-c6fd58b1fc96", "tag": "715e7be7-dd5f-443a-b42e-d143d247ea7d"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:35.446589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-f83b3bb2-5bbd-4fa9-be77-a62d7c2f855e req-d84f5613-7b0d-4a0f-a5a5-558bf43b5fad service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.458753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:35.463225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4b94495c-2d5a-4873-a70b-9566260ceb70 req-e522af3d-4d7d-490d-a1d0-e96a4aa380e4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.463744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4b94495c-2d5a-4873-a70b-9566260ceb70 req-e522af3d-4d7d-490d-a1d0-e96a4aa380e4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.487217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-f83b3bb2-5bbd-4fa9-be77-a62d7c2f855e req-d84f5613-7b0d-4a0f-a5a5-558bf43b5fad service nova] Creating event network-changed:67324695-43bc-444a-9a35-3a3bbb87de0b for instance 6b566c6c-a418-4384-b99e-c7295e754e92 on hv-cinder-84902 Jul 20 14:49:35.506539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-f83b3bb2-5bbd-4fa9-be77-a62d7c2f855e req-d84f5613-7b0d-4a0f-a5a5-558bf43b5fad service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.114002 Jul 20 14:49:35.507302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-4b94495c-2d5a-4873-a70b-9566260ceb70 req-e522af3d-4d7d-490d-a1d0-e96a4aa380e4 service nova] Creating event network-changed:715e7be7-dd5f-443a-b42e-d143d247ea7d for instance 52220445-7cc1-4635-9566-c6fd58b1fc96 on hv-cinder-84902 Jul 20 14:49:35.512214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 144/295] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:35.518526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-4b94495c-2d5a-4873-a70b-9566260ceb70 req-e522af3d-4d7d-490d-a1d0-e96a4aa380e4 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080960 Jul 20 14:49:35.522182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 152/296] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:35.530518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:35.533641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:35.535351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:35.538267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:35.561392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.562224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.563987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.564442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.565981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.566426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.568933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:35.569389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:35.715954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:35.963747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:35.977950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8064b809-9bc5-4b3a-933b-29a8526bedeb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.463982 Jul 20 14:49:35.979284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 145/297] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:35 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:36.018022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:36.027776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-de6db812-315f-47d6-868c-57ddf1a7e96e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.502551 Jul 20 14:49:36.038089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 153/298] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:35 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:37.000974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:37.002041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:37.011917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:37.012700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:37.013982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:37.014693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:37.045779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:37.046496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:37.048857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:37.056472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:37.057330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:37.058983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:37.059543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:37.245354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:37.253337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b9988589-5397-4cf9-8bd8-a2fa888ba00f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.259326 Jul 20 14:49:37.254448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 146/299] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:36 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:37.264994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:37.273562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0beb343-bcfe-4910-a4ba-226c5a5b9c63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.231895 Jul 20 14:49:37.283528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 154/300] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:37 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:37.544666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-2f0aa148-bb10-4890-b672-1c0b3f636725 req-6a44ab65-8de6-4274-be74-e757a4e9ecb8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f3521d1-81cb-4217-b43a-b84b94911839", "tag": "ae0ff63e-d9ee-41cd-875c-523dfce9bd0c"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:37.555083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-2f0aa148-bb10-4890-b672-1c0b3f636725 req-6a44ab65-8de6-4274-be74-e757a4e9ecb8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:37.555555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-2f0aa148-bb10-4890-b672-1c0b3f636725 req-6a44ab65-8de6-4274-be74-e757a4e9ecb8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:37.575480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-2f0aa148-bb10-4890-b672-1c0b3f636725 req-6a44ab65-8de6-4274-be74-e757a4e9ecb8 service nova] Creating event network-changed:ae0ff63e-d9ee-41cd-875c-523dfce9bd0c for instance 2f3521d1-81cb-4217-b43a-b84b94911839 on hv-cinder-84902 Jul 20 14:49:37.580503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-2f0aa148-bb10-4890-b672-1c0b3f636725 req-6a44ab65-8de6-4274-be74-e757a4e9ecb8 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039458 Jul 20 14:49:37.581102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 147/301] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:38.272973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:38.273625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:38.281921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:38.282371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:38.283154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:38.283584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:38.287816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:38.292341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:38.293116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:38.300752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:38.301348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:38.302226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:38.302793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:38.494397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:38.502248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7d52a1f-a717-41a2-a31c-1bc9defb1faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.233953 Jul 20 14:49:38.511818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:38.512066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 155/302] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:38 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:38.518445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6e4421ad-24ac-43e5-a9be-d4dc5ffc0c70 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.230966 Jul 20 14:49:38.519451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 148/303] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:38 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:39.518558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:39.519140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:39.525823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:39.526267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:39.527028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:39.527491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:39.535936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:39.536542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:39.543503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:39.544026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:39.544734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:39.545166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:39.763900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:39.763900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-77b4f11a-f09a-4c06-b992-19ff34a2c664 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.245412 Jul 20 14:49:39.770074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 156/304] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:39 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:39.770943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:39.779730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd4ba447-1dfe-4b98-a5e8-2aa44f61da5a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.247649 Jul 20 14:49:39.780804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 149/305] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:39 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:40.781951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:40.782888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:40.793302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:40.793830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:40.794981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:40.795736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:40.803029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:40.803808 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:40.814047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:40.814751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:40.816027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:40.816724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:41.013151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:41.022263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0512dd6f-aa73-449a-8967-0499f545da12 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.247836 Jul 20 14:49:41.022848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 157/306] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:40 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:41.023922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:41.035930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8416998f-085e-4444-ba68-ebd513b0d095 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.239980 Jul 20 14:49:41.036689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 150/307] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:40 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:42.044370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:42.044994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:42.052100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:42.052557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:42.053334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:42.053741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:42.054243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:42.054343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:42.064232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:42.064669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:42.065443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:42.065867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:42.254802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:42.263403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:42.263574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1185ca16-7188-4c04-8521-eba50efa5e55 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.222992 Jul 20 14:49:42.264165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 158/308] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:42 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:42.272233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f53564cd-5dd1-42cd-8d24-d3ad9212218e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.222545 Jul 20 14:49:42.272850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 151/309] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:42 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:43.285259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:43.286262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:43.292802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:43.293412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:43.294201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:43.294656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:43.295463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:43.295897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:43.301232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:43.301774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:43.302756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:43.303477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:43.496938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:43.500131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:43.504232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:43.505194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1fd6bcb0-f97f-4351-8fd0-f58777cf6e4d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.226829 Jul 20 14:49:43.505876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 159/310] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:43 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:43.510390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-81506374-d97e-4635-99fe-ea486b9f5171 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.222945 Jul 20 14:49:43.511204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 152/311] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:43 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:44.522153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:44.522772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:44.526871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:44.528114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:44.529616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:44.530539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:44.531609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:44.532023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:44.535609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:44.536192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:44.536981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:44.537416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:44.760168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:44.764328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:44.768701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-517954af-fbfa-4d8b-b4db-d89719ed054e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.250463 Jul 20 14:49:44.769249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 160/312] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:44 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:44.772135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a1e0709a-fe03-4744-b377-147cd50f41ad tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.249025 Jul 20 14:49:44.772931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 153/313] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:44 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:45.786265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:45.787445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:45.791685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:45.792622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:45.803652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:45.804294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:45.805191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:45.805812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:45.815080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:45.815573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:45.816316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:45.816752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:45.965782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:45.973739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ae0e63ca-fcdb-4c51-afef-5042614e472f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.191201 Jul 20 14:49:45.974326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 161/314] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:45 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:46.105007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:46.113305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e6420f2e-85b9-468c-8721-9053c8c23dd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.327164 Jul 20 14:49:46.113883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 154/315] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:45 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:46.990302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:46.990889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:46.997131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:46.997550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:46.998323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:46.999082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:47.132533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:47.133564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:47.143127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:47.143587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:47.144431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:47.144869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:47.145400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:47.156509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-686d0d63-4be8-4c0a-963f-53694efc1137 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.170248 Jul 20 14:49:47.171996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 162/316] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:46 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:47.317846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:47.326499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8ccb20c2-6c11-4413-829e-0f8f6f99a48e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.198820 Jul 20 14:49:47.327103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 155/317] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:47 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:48.178541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:48.179280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:48.186479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.187273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.188395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.189855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.263811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:48.264362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:48.273527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.273981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.274759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.275221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.317563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:49:48.466928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6371651e-54d2-40ec-acd8-3905f8299914 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "DELETE /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 204 len: 0 microversion: 2.1 time: 0.206666 Jul 20 14:49:48.471735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 156/318] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:49:48 2022] DELETE /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:49:48.480207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:48.481264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:48.491899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.492625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.494270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.494912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.538888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:48.548656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4117354d-1c51-4cdf-8b0b-7f6a84a1fc9f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.374080 Jul 20 14:49:48.550142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 163/319] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:48 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:48.557821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:48.559467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:48.566672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.567407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.568359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:48.568945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:48.715560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:48.723404 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cebbcae3-a985-4fee-baa6-354163360e80 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.250125 Jul 20 14:49:48.724012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 157/320] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:48 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:48.764336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:48.772878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-327f221c-d552-40e5-bd83-6d7e263b6a7b tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.220381 Jul 20 14:49:48.773429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 164/321] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:48 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:49.566547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:49.567158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:49.574893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.575379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.576161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.576588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.721502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:49.729303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2aa543b3-d1f3-4616-bc62-123fdf50f014 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1279 microversion: 2.1 time: 0.166958 Jul 20 14:49:49.730247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 158/322] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:49 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1279 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:49.742556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:49.743342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:49.750096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.750646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.751517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.752020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.791151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:49.791764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:49.798156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.798602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.799402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:49.799870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:49.937310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:49.945827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-60a843aa-0885-4738-9813-99b9ab3b6fbe tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.208333 Jul 20 14:49:49.946517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 165/323] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:49 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:49.972527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:49.980949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3e0fda97-fc61-4fa3-96fd-293dcdac6162 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.193718 Jul 20 14:49:49.981497 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 159/324] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:49 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:50.746061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:50.746860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:50.753202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:50.753820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:50.754742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:50.755192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:50.917253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:50.925930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30bbcd37-900a-4800-9fad-0b71621eccac tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.183787 Jul 20 14:49:50.926685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 166/325] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:50 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:50.965190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:50.966185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:50.977039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:50.977979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:50.979255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:50.979959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:50.999452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:51.000315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:51.006572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:51.007123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:51.007889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:51.008320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:51.128015 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:51.136645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4eb7898b-8b73-46e9-b0e1-d5e82afba7f9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.177554 Jul 20 14:49:51.137046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 160/326] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:50 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:51.282168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:51.292803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-14d5f509-63e2-4f37-aed3-76416937b9a4 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.296900 Jul 20 14:49:51.293450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 167/327] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:50 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:51.943864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:51.944507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:51.951280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:51.951722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:51.952495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:51.952926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:52.097176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:52.105572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ef14a240-19c4-46ad-9dec-227b17ea9497 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.165619 Jul 20 14:49:52.106158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 161/328] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:51 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:52.153918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:52.154525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:52.166814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:52.168713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:52.169631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:52.169994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:52.312587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:52.314127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:52.321845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:52.323247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:52.325065 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:52.325532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:52.612984 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:52.613354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:52.621712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1a128b7c-7791-4a62-98b7-e0d36f5adea1 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.313371 Jul 20 14:49:52.622427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 162/329] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:52 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:52.631563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f15c04ff-40c6-4e52-a181-c8aceec228a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.477415 Jul 20 14:49:52.634474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 168/330] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:52 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:53.122010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:53.122752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:53.129686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.130137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.130884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.131347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.329380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:53.337564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-82fc608d-2694-4c0f-822c-cc7d516a593a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.219429 Jul 20 14:49:53.338146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 163/331] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:53.638552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:53.639330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:53.645484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:53.646053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.646220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:53.646504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.647308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.647742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.653018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.653520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.654364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:53.655228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:53.868786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:53.876993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d78e283d-a6a9-4cff-850f-09d0a217972a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.235932 Jul 20 14:49:53.877575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 164/332] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:53 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:53.884527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:53.892725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b24c8796-c5dd-41a6-b031-b37b9f38f8da tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.257925 Jul 20 14:49:53.893304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 169/333] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:53 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:54.354403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:54.355128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:54.365782 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.366244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:54.367024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.367490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:54.509129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:54.520076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-04ab8fbe-0610-46bc-829d-fb938a753ebc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.169062 Jul 20 14:49:54.521070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 165/334] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:54.893855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:54.895010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:54.901983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.901983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:54.902720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.903194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:54.910251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:54.911093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:54.917558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.918096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:54.919191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:54.919330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:55.121071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:55.124265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:55.131853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-913b6049-cb90-4a19-9685-10c0d8191e9e tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.224959 Jul 20 14:49:55.132423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eaaa2f9c-3b38-46e3-9625-a3c4a8e23125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.242551 Jul 20 14:49:55.133227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 166/335] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:54 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:55.142234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 170/336] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:54 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:55.537993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:55.538622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:55.546096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:55.546547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:55.547406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:55.547848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:55.753748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:55.758374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:55.764264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b917efdf-1e2e-4b47-8460-1f3e3e2cdffd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.230105 Jul 20 14:49:55.764971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 167/337] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:55 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:56.153178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:56.153789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:56.160027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:56.160651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:56.161059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.161777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.162653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.163181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.167399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.168034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.168807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.169562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.368866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:56.370560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:56.372890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:56.379728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a09412a8-ab3d-4643-a431-86e49768b23e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.233973 Jul 20 14:49:56.380494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 171/338] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:56 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:56.380600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6d142bfb-4545-43e5-8ecc-08a380ce51d5 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.232365 Jul 20 14:49:56.380879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 168/339] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:56 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:56.782335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:56.783008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:56.790414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.790860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.791736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:56.792206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:56.954298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:56.962744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ffcd94cf-0572-42b9-9398-104eb69ab8d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.184734 Jul 20 14:49:56.963566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 172/340] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:56 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:57.397920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:57.398537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:57.400663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:57.401311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:57.409886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.410339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:57.410487 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.410844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:57.411157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.411663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.411806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:57.412098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:57.608719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:57.615348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:57.617014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-287b83d0-bc9a-4baa-b96f-3b021098fe1a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.222986 Jul 20 14:49:57.617590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 169/341] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:57 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:57.623730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c017c7fc-d655-461e-af23-869a459c3919 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.226963 Jul 20 14:49:57.624279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 173/342] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:57 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:57.980122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:57.980839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:57.982776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=87,nova_api:UPDATE=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:49:57.988536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.989019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:57.989869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:57.990295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:58.131557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:58.139387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8f760484-9276-4cd2-83c3-1b22460cc5df tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.162987 Jul 20 14:49:58.139979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 170/343] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:57 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:58.634062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:58.634619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:58.640316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:58.641213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:58.641604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:58.641716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:58.642593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:58.643027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:58.648323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:58.648767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:58.649538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:58.649968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:58.894565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:58.899374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:58.903423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c65dd6ed-e648-4294-8cd1-3062634f3e29 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.273025 Jul 20 14:49:58.904351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 174/344] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:58 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:58.918543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6a48414c-afeb-412d-a698-4b44ab2f7abf tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1553 microversion: 2.1 time: 0.281961 Jul 20 14:49:58.928717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 171/345] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:58 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1553 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:59.087426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-291663302", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:59.147453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] validate_networks() for None {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:49:59.157790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:59.159274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:59.165644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.166084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.167094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.167715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.574753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:59.583203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1024c65-f37c-401f-bb5f-c02e5fac9faa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.429484 Jul 20 14:49:59.584189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 172/346] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:59 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:49:59.688641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:49:59.689643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:49:59.690280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:49:59.690687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:49:59.691131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:49:59.700634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.701053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.702043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.702481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.721333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Getting quotas for project 50a5f7380c804eba98ab4a8ee65dc1d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:49:59.725693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Getting quotas for user e1750ad6625e4cc0b430a81ed188e538 and project 50a5f7380c804eba98ab4a8ee65dc1d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:49:59.737521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:49:59.739902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:49:59.803741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-998455e7-9db1-462a-b118-900f810a60df tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.720208 Jul 20 14:49:59.805275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 175/347] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:49:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 722 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:49:59.816654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9bd022b5-c0ac-4d54-bfa2-7ba5c3965a9e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:59.817517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9bd022b5-c0ac-4d54-bfa2-7ba5c3965a9e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:59.854040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-e55377f6-cedf-4f27-8cf5-ce9f98401f04 req-a9ebf99e-6a17-4f51-b438-b543aaf00c5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ad8269f-e328-4587-a622-d5940cd92fa7", "name": "network-vif-deleted", "tag": "e938812d-42ef-47c6-9b5a-d4cab1fca9f5"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:49:59.863958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-e55377f6-cedf-4f27-8cf5-ce9f98401f04 req-a9ebf99e-6a17-4f51-b438-b543aaf00c5d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.864637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-e55377f6-cedf-4f27-8cf5-ce9f98401f04 req-a9ebf99e-6a17-4f51-b438-b543aaf00c5d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.881772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bd022b5-c0ac-4d54-bfa2-7ba5c3965a9e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:49:59.889416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9bd022b5-c0ac-4d54-bfa2-7ba5c3965a9e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1214 microversion: 2.1 time: 0.077927 Jul 20 14:49:59.890508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 173/348] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:59 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1214 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:49:59.890936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-e55377f6-cedf-4f27-8cf5-ce9f98401f04 req-a9ebf99e-6a17-4f51-b438-b543aaf00c5d service nova] Creating event network-vif-deleted:e938812d-42ef-47c6-9b5a-d4cab1fca9f5 for instance 5ad8269f-e328-4587-a622-d5940cd92fa7 on hv-cinder-84902 Jul 20 14:49:59.900767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-e55377f6-cedf-4f27-8cf5-ce9f98401f04 req-a9ebf99e-6a17-4f51-b438-b543aaf00c5d service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050750 Jul 20 14:49:59.901583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 176/349] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:49:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:49:59.921221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:59.921846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:59.932954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.933631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.934693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.935371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.936365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:49:59.937075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:49:59.944138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.944841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:49:59.945830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:49:59.946424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:00.030443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:00.042468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-418873b9-54c0-482e-b73f-62b4a5b9e097 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.110209 Jul 20 14:50:00.043526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 177/350] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:59 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:00.152175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:00.163541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cf16868e-6621-4318-b1ef-1316311edd0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.245744 Jul 20 14:50:00.164371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 174/351] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:49:59 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:00.601455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:00.602051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:00.609634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:00.610083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:00.610844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:00.611313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:00.751892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:00.760155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-291dad96-733d-4a0a-9ba5-ed2766c0e29a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.162700 Jul 20 14:50:00.760736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 178/352] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:00 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:00.908704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:00.909311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:00.916866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:00.917313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:00.918083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:00.918504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:00.982864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:00.990985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2a21fc95-e30c-4698-a5ac-748c7d84f9e8 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1218 microversion: 2.1 time: 0.086974 Jul 20 14:50:00.991861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 175/353] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:00 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1218 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:01.060159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:01.060787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:01.067676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.068158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.068944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.069375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.135454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:01.145995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-beda2ac3-de48-48cf-9862-dc0a25b25556 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.089691 Jul 20 14:50:01.146573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 179/354] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:01 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:01.182014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:01.182645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:01.189572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.190032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.190829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.191311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.326365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:01.338831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d913363e-4c11-4da4-ac6b-a5c3e9b1065a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.161114 Jul 20 14:50:01.339434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 176/355] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:01 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:01.777518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:01.778303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:01.785461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.785910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.786735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:01.787180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:01.933998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:01.942879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a7e9a771-2b8f-4217-9be0-7b06532a8387 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.168947 Jul 20 14:50:01.943759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 180/356] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:01 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:02.013332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:02.014183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:02.022313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.022774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.023634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.024108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.093389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:02.101665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9bb08b20-6c06-4f87-8cdc-916dbb2af2ac tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1218 microversion: 2.1 time: 0.095148 Jul 20 14:50:02.102244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 177/357] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:02 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1218 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:02.163712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:02.164344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:02.172105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.172612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.174068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.174925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.239338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:02.247998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e87bbda0-2b5e-4706-9c82-44b65249db1f tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.088085 Jul 20 14:50:02.248580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 181/358] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:02 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:02.356723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:02.357366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:02.364898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.365347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.366165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.366595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.507758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:02.515739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-91f955ab-f3fe-4dde-bc1e-0dc889d44aa9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.163007 Jul 20 14:50:02.516310 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 178/359] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:02 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:02.960565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:02.961139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:02.967882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.968325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:02.969087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:02.969512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.112379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:03.118381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:03.118918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:03.121277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f3fe2941-cc7f-4fb4-806f-542f3b822cf4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.164807 Jul 20 14:50:03.121854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 182/360] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:02 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:03.125923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.126373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.127203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.127639 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.207369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:03.215355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5d3063f5-7312-4f49-9148-c6375848435e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1210 microversion: 2.1 time: 0.101103 Jul 20 14:50:03.215931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 179/361] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:03 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1210 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:03.265330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:03.265915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:03.272787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.273231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.273995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.274419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.342193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:03.350500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63fa877a-5691-417e-b3d9-205933b6c1fb tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.089054 Jul 20 14:50:03.351078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 183/362] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:03 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:03.533917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:03.534630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:03.541610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.542188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.543425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:03.543887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:03.679151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:03.688928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2b36fed3-e4c8-460e-839b-868a2d3c5f2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.158884 Jul 20 14:50:03.689709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 180/363] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:03 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:04.137967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:04.138584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:04.146713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.147776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.147889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.148356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.232927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:04.233637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:04.240706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.241160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.241946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.242376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.309538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:04.318212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-99478249-b20c-4e53-a93b-2ee6c4595c19 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.183996 Jul 20 14:50:04.318771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 184/364] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:04 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:04.326221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:04.334253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-12722597-631a-46da-aa6c-4b8bea5a5bc9 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1210 microversion: 2.1 time: 0.105231 Jul 20 14:50:04.334798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 181/365] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:04 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1210 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:04.368058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:04.368668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:04.375380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.375827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.376591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.377084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.442605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:04.453104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24cd1885-3234-4a81-88e2-bd5e45322d78 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.088842 Jul 20 14:50:04.453681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 185/366] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:04 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:04.706320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:04.706932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:04.715123 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.716004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.716455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:04.717018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:04.848477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:04.856779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a1279707-f481-42db-9253-030f90ab2e48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.154293 Jul 20 14:50:04.857354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 182/367] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:04 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:05.335584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:05.336159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:05.342652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.343151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.344445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.345230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.353803 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:05.354841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:05.364119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.364549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.365344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.365776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.684219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:05.691832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fd6d61eb-dfc1-418a-b9f0-dc2b05c362ad tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1266 microversion: 2.1 time: 0.343852 Jul 20 14:50:05.692432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 183/368] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:05 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1266 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:05.699316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:05.699930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:05.707935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.707935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.707935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.707935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.748159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:05.756155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56e6312e-ec8f-427e-8c6e-db6b38da5a5c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.424554 Jul 20 14:50:05.756728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 186/369] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:05 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:05.781125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:05.790649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cde9b2d0-d435-409a-aeb6-5094e70d09a8 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.095140 Jul 20 14:50:05.791260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 184/370] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:05 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:05.877975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:05.878584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:05.880584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:05.886242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.886712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:05.887574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:05.888007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.042027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:06.049955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84462b82-16fd-437d-8f6a-33ad4c098998 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.176123 Jul 20 14:50:06.050521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 187/371] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:05 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:06.707951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:06.708563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:06.715759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.716206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.716986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.717408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.736969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:06.773916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:06.774772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:06.782014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.782232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:06.782465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.783393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.783823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.790236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c583d12d-6f83-4d89-a464-b9bd91ea9fbe tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1266 microversion: 2.1 time: 0.086174 Jul 20 14:50:06.790830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 185/372] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:06 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1266 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:06.817542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:06.818150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:06.825134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.825134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.825801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:06.826232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:06.902137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:06.910718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-03f4a7a7-e86f-45fc-b04e-1322144e24b0 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1345 microversion: 2.1 time: 0.096878 Jul 20 14:50:06.916579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 186/373] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:06 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1345 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:06.952537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:06.960492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ec7bc06-0943-4aeb-9d7f-c5ed3a58dff3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.190879 Jul 20 14:50:06.961265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 188/374] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:06 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:07.066303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:07.066911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:07.073240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.073692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.074468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.074907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.216284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:07.224149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-498b2a51-820b-4984-935d-514e3a2c1131 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.161723 Jul 20 14:50:07.225382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 187/375] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:07 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:07.807659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:07.808316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:07.814850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.815330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.816104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.816534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.884239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:07.887791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:07.892821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5e5fa63b-9cb6-4d94-8827-652c542bcf53 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1274 microversion: 2.1 time: 0.088973 Jul 20 14:50:07.893424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 189/376] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:07 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1274 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:07.928417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:07.929306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:07.936160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.936602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.937658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.938377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.975068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.975547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.976000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lazy-loading 'fault' on Instance uuid 5ad8269f-e328-4587-a622-d5940cd92fa7 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:07.977760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:07.978371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:07.984609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.985051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:07.985812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:07.986385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:08.023669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:08.032106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd00fa86-b368-44be-9c29-6b9462732b26 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1350 microversion: 2.1 time: 0.108032 Jul 20 14:50:08.032684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 188/377] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:07 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1350 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:08.122488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:08.132322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5b355210-f052-4d5d-8db6-d044fb6c11fd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.158530 Jul 20 14:50:08.132876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 190/378] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:07 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:08.242377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:08.243014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:08.249718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:08.250150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:08.250931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:08.251431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:08.407899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:08.415937 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4d6dbcb7-61a0-4d3c-b5fa-3853fa4709ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.177810 Jul 20 14:50:08.416480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 189/379] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:08 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:08.911137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:08.911701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:08.918353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:08.918798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:08.919584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:08.920011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:08.985041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:08.992969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-caeb8b24-2da9-4e4c-acbc-c3a80e0832c3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1274 microversion: 2.1 time: 0.085822 Jul 20 14:50:08.993494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 191/380] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:08 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1274 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:09.051764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:09.052695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:09.062594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.063740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.064246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.064904 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.101532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.101868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.102339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lazy-loading 'fault' on Instance uuid 5ad8269f-e328-4587-a622-d5940cd92fa7 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:09.142582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:09.150965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-69809a39-0a57-4a35-a3b7-5b4257b84d0d tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1350 microversion: 2.1 time: 0.104728 Jul 20 14:50:09.151607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 190/381] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:09 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1350 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:09.153191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:09.154625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:09.164610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.165061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.165847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.166284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.306025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:09.313847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b5c106d0-da2a-42fd-b486-fa5246c2a454 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.167711 Jul 20 14:50:09.314486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 192/382] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:09 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:09.433253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:09.433846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:09.440350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.440795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.441553 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:09.441981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:09.623173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:09.631807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1ea18a0c-bccb-4e4b-bab1-cccda8df187b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.202292 Jul 20 14:50:09.632650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 191/383] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:09 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:10.011589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:10.012174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:10.014048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:10.014708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:10.020800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.021219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.021986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.022418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.085247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:10.093391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-886b402d-2721-4c2d-bed6-c20b66dfeace tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1284 microversion: 2.1 time: 0.085635 Jul 20 14:50:10.093971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 193/384] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:10 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1284 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:10.168348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:10.168983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:10.170870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:10.178010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.178565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.179469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.179908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.212299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.212788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.213213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lazy-loading 'fault' on Instance uuid 5ad8269f-e328-4587-a622-d5940cd92fa7 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:10.256711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:10.265391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c23dedec-0bf0-465e-b678-c535031185bd tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1350 microversion: 2.1 time: 0.100820 Jul 20 14:50:10.266056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 192/385] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:10 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1350 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:10.332966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:10.333859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:10.340424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.340876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.341643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.342071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.486059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:10.497292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4385354-2908-4d2b-bc1c-1a0bcd75cd0d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.168517 Jul 20 14:50:10.498107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 194/386] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:10 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:10.649590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:10.650198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:10.657264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.657830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.658799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:10.659274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:10.805767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:10.813418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-75d85520-40ca-4097-9ac3-7c1b8ec852bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.167793 Jul 20 14:50:10.814019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 193/387] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:10 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:11.114233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:11.115186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:11.125455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.126166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.127331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.127764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.191987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:11.203932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce5f5074-1647-4ae3-9880-0f66b83ee069 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1284 microversion: 2.1 time: 0.095838 Jul 20 14:50:11.204837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 195/388] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:11 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:11.284678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:11.285328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:11.292326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.292771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.293530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.293946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.326504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.326949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.327437 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lazy-loading 'fault' on Instance uuid 5ad8269f-e328-4587-a622-d5940cd92fa7 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:11.371911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5373c0299ea64c9f872e3bc04bfb0b74', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d28cd25ee0c476e8ab3b24d7f2f9ab9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:11.379993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-11c40aa9-7880-432d-bab0-79366888f129 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 200 len: 1350 microversion: 2.1 time: 0.099333 Jul 20 14:50:11.380560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 194/389] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:11 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 1350 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:11.517133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:11.517754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:11.524549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.524999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.525772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.526232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.669286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:11.677092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-94413346-04c8-4fc9-a055-d9059352a2de tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.163855 Jul 20 14:50:11.677683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 196/390] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:11 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:11.830174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:11.830934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:11.837833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.838282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:11.839041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:11.839506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.068132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:12.076189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ccf4379c-0afd-4e10-a580-0e8d2f1f23f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.249942 Jul 20 14:50:12.076754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 195/391] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:11 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:12.227185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:12.227747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:12.234417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.234859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.235791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.236296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.305298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:12.313608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d59f0c49-e484-485a-af0d-fdddbcdc3225 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1272 microversion: 2.1 time: 0.090436 Jul 20 14:50:12.314190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 197/392] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:12 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1272 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:12.397306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:12.398242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:12.406319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.406760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.407568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.407991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.424523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] HTTP exception thrown: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Request method failure captured: Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 HTTP/1.1 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWfy3UP_1BLpST0LkI1Zz2WEKdhcKNpDE9f1eT0kSLtLJ8grFQvQniMUZBp_e_IXo6u7aF5OfHLFoOJiE3p2Fi518YcbnqXMuO3jSx2GaPB6JjFIo1latCMQtF6JqRdUdpqkiT-8y3xurz95GJVFobM9BTkIH4Srsi39FLOVkNhDB0 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:50:12.434783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}] Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 5373c0299ea64c9f872e3bc04bfb0b74 Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.436462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '5ad8269f-e328-4587-a622-d5940cd92fa7'} Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:12.437901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.439507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.441254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Returning 404 to user: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:12.441254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8010b54c-bf86-4762-b7d7-40b2f7804bce tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 404 len: 111 microversion: 2.1 time: 0.043269 Jul 20 14:50:12.441254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 196/393] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:12 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:12.556715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:12.557720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:12.563993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.564509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.565325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.565783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.579025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] HTTP exception thrown: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Request method failure captured: Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7/os-volume_attachments HTTP/1.1 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWfy3UP_1BLpST0LkI1Zz2WEKdhcKNpDE9f1eT0kSLtLJ8grFQvQniMUZBp_e_IXo6u7aF5OfHLFoOJiE3p2Fi518YcbnqXMuO3jSx2GaPB6JjFIo1latCMQtF6JqRdUdpqkiT-8y3xurz95GJVFobM9BTkIH4Srsi39FLOVkNhDB0 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:12.583107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}] Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5373c0299ea64c9f872e3bc04bfb0b74 Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.584387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '5ad8269f-e328-4587-a622-d5940cd92fa7'} Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:12.588374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:12.589746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:12.590958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Returning 404 to user: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:12.590958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-83ba7dba-ae51-42f1-8be6-c0b708167087 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.031979 Jul 20 14:50:12.590958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 198/394] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 14:50:12 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7/os-volume_attachments => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:12.705126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:12.705757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:12.713410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.714246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.715280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:12.716144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:12.959376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:12.969327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-50c07cb1-e0ec-442b-a296-39c0e1267b35 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.269271 Jul 20 14:50:12.970287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 197/395] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:12 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:13.096167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:13.096978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:13.106690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.107327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.108280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.108712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.246486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:13.254724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c0f24f0-69d9-4ec8-bd05-9467274c058a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.164340 Jul 20 14:50:13.255579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 199/396] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:13 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:13.330347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:13.331029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:13.337826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.338265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.339042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.339527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.492874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:13.513797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f146d30b-4aa4-468c-ab48-533dad91e379 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.187063 Jul 20 14:50:13.514831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 198/397] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:13 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:13.826653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:13.827450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:13.835267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.835725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.836501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.836942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.851137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] HTTP exception thrown: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Request method failure captured: Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 HTTP/1.1 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWfy3UP_1BLpST0LkI1Zz2WEKdhcKNpDE9f1eT0kSLtLJ8grFQvQniMUZBp_e_IXo6u7aF5OfHLFoOJiE3p2Fi518YcbnqXMuO3jSx2GaPB6JjFIo1latCMQtF6JqRdUdpqkiT-8y3xurz95GJVFobM9BTkIH4Srsi39FLOVkNhDB0 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:13.853425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}] Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5373c0299ea64c9f872e3bc04bfb0b74 Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.854852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '5ad8269f-e328-4587-a622-d5940cd92fa7'} Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:13.855767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.857230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.858808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Returning 404 to user: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:13.858808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-35e75a6c-37e7-4a11-8a4e-cb890d5c3082 tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "DELETE /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 404 len: 111 microversion: 2.1 time: 0.033495 Jul 20 14:50:13.858808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 200/398] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:50:13 2022] DELETE /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:13.864020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:13.864679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] [instance: 5ad8269f-e328-4587-a622-d5940cd92fa7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:13.871085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.871556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.872349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.872779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.886739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] HTTP exception thrown: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Request method failure captured: Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 HTTP/1.1 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWfy3UP_1BLpST0LkI1Zz2WEKdhcKNpDE9f1eT0kSLtLJ8grFQvQniMUZBp_e_IXo6u7aF5OfHLFoOJiE3p2Fi518YcbnqXMuO3jSx2GaPB6JjFIo1latCMQtF6JqRdUdpqkiT-8y3xurz95GJVFobM9BTkIH4Srsi39FLOVkNhDB0 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:50:13.888303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/3d28cd25ee0c476e8ab3b24d7f2f9ab9"}]}] Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 3d28cd25ee0c476e8ab3b24d7f2f9ab9 Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-DeleteServersTestJSON-1197471686 Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 5373c0299ea64c9f872e3bc04bfb0b74 Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-DeleteServersTestJSON-1197471686-project Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.889545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '5ad8269f-e328-4587-a622-d5940cd92fa7'} Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:13.890283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. Jul 20 14:50:13.891449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:13.892751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] Returning 404 to user: Instance 5ad8269f-e328-4587-a622-d5940cd92fa7 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:13.892751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cc53969b-a094-4449-9a0f-5258a660835c tempest-DeleteServersTestJSON-1197471686 tempest-DeleteServersTestJSON-1197471686-project] 10.222.0.15 "GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7" status: 404 len: 111 microversion: 2.1 time: 0.029835 Jul 20 14:50:13.892751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 199/399] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:13 2022] GET /compute/v2.1/servers/5ad8269f-e328-4587-a622-d5940cd92fa7 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:13.989076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:13.989802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:13.996073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.996603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:13.997425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:13.997859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.236328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:14.244688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef99ab4a-81e2-41ed-b090-c696da50ab38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.260357 Jul 20 14:50:14.245265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 201/400] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:13 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:14.272470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:14.273204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:14.280207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:14.280879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.281736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:14.282240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.531612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:14.532365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:14.536573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:14.538908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:14.539392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.540139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:14.540763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.544192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3625eba2-b70a-47eb-bbf2-504fd6ce5246 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.275918 Jul 20 14:50:14.544882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 200/401] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:14 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:14.783799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:14.813460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-114eed9c-ad54-418b-bee7-d20dd6f5c553 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.285879 Jul 20 14:50:14.813940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 202/402] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:14 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:14.963382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-06b1102d-da50-427a-8b73-ebbd42048b13 req-25248d97-373c-4c96-8e86-3ba6fc6bec48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f3521d1-81cb-4217-b43a-b84b94911839", "name": "network-vif-plugged", "status": "completed", "tag": "ae0ff63e-d9ee-41cd-875c-523dfce9bd0c"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:14.971959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-06b1102d-da50-427a-8b73-ebbd42048b13 req-25248d97-373c-4c96-8e86-3ba6fc6bec48 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:14.972392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-06b1102d-da50-427a-8b73-ebbd42048b13 req-25248d97-373c-4c96-8e86-3ba6fc6bec48 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:14.983708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-06b1102d-da50-427a-8b73-ebbd42048b13 req-25248d97-373c-4c96-8e86-3ba6fc6bec48 service nova] Creating event network-vif-plugged:ae0ff63e-d9ee-41cd-875c-523dfce9bd0c for instance 2f3521d1-81cb-4217-b43a-b84b94911839 on hv-cinder-84902 Jul 20 14:50:14.989309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-06b1102d-da50-427a-8b73-ebbd42048b13 req-25248d97-373c-4c96-8e86-3ba6fc6bec48 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029827 Jul 20 14:50:14.990305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 201/403] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:50:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:15.267559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:15.268249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:15.287121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:15.287933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:15.290113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:15.291872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:15.563348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:15.564366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:15.575518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:15.576191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:15.577352 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:15.577972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:15.986353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:15.990207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:15.995263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-93ce9810-d3a3-459a-9319-8283ae871fe2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.437306 Jul 20 14:50:15.995920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 202/404] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:15 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:16.000501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:16.001105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:16.007936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:16.008386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:16.009187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:16.009622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:16.016652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:16.026628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1ff15ffc-9cd7-440a-972d-35334c86a7ef tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.762772 Jul 20 14:50:16.027312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 203/405] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:15 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:16.312738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:16.338869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-41b1f416-2387-4d4d-9e59-e46fc53acf26 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.332147 Jul 20 14:50:16.338869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 203/406] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:15 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:16.731673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-b7d5eec8-e199-452f-bb5a-425ebf1710d1 req-29857c53-eabe-4cd9-b139-f86dfd0b2daa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38c6f67e-423e-44d8-abcb-73298f72167b", "tag": "9e75b368-eeb6-4a49-897c-6366158a4049"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:16.747265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b7d5eec8-e199-452f-bb5a-425ebf1710d1 req-29857c53-eabe-4cd9-b139-f86dfd0b2daa service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:16.749198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b7d5eec8-e199-452f-bb5a-425ebf1710d1 req-29857c53-eabe-4cd9-b139-f86dfd0b2daa service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:16.777700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-b7d5eec8-e199-452f-bb5a-425ebf1710d1 req-29857c53-eabe-4cd9-b139-f86dfd0b2daa service nova] Creating event network-changed:9e75b368-eeb6-4a49-897c-6366158a4049 for instance 38c6f67e-423e-44d8-abcb-73298f72167b on hv-cinder-84902 Jul 20 14:50:16.794882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-b7d5eec8-e199-452f-bb5a-425ebf1710d1 req-29857c53-eabe-4cd9-b139-f86dfd0b2daa service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067275 Jul 20 14:50:16.796140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 204/407] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:50:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:17.013087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:17.013998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:17.021275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.021725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.022565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.023066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.044503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:17.048024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:17.055402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.056179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.057173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.058186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.250240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:17.258076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:17.265933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-343aba31-db51-40da-87cf-0ad7f8bea4f6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.256590 Jul 20 14:50:17.266656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b54687c6-ae17-4659-95c4-23513ef4dda8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1267 microversion: 2.1 time: 0.226210 Jul 20 14:50:17.267284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 204/408] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:17 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:17.267731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 205/409] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:17 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1267 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:17.346587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:17.347460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:17.354978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.355726 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.356926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:17.357598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:17.598434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:17.607429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fd175c6e-626a-4453-bb37-034343e8b67a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.265245 Jul 20 14:50:17.608204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 205/410] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:17 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:18.289389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:18.289943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:18.290293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:18.290406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:18.300519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.301174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.304031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.304757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.305633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.311394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.312655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.315853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.713937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:18.735958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a1b925bb-3dd9-46d7-8097-21c3f4a23726 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1521 microversion: 2.1 time: 0.451871 Jul 20 14:50:18.737002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 206/411] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:18 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1521 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:18.739659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:18.744707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:18.746555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:18.752017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5f0cc951-46cf-4eb6-b084-d457e5d89ec8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1455 microversion: 2.1 time: 0.467289 Jul 20 14:50:18.752728 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 206/412] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:18 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1455 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:18.754628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.755639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.757341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.758485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.771792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:18.772600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:18.779502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.780192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:18.781145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:18.781638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:19.094893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:19.110136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:19.123103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-018fde1e-d927-4c55-a1ea-82f582ae37b3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1455 microversion: 2.1 time: 0.354350 Jul 20 14:50:19.125263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 207/413] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:18 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1455 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:19.140662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0143d271-cdb6-495e-9983-b713f67cc435 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.400683 Jul 20 14:50:19.141899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 207/414] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:18 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:19.754893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:19.756087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:19.764720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:19.765169 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:19.767725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:19.768699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:19.962532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:19.977492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-754af5c9-b429-4800-818d-87deea28e84f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1501 microversion: 2.1 time: 0.227789 Jul 20 14:50:19.978069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 208/415] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:19 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1501 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:19.997088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:19.998211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:20.012450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:20.012907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:20.013901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:20.014525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:20.049976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:20.158267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:20.159039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:20.169695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:20.170154 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:20.171034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:20.171496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:20.383230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:20.395175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-97d25c7f-22ae-43c8-af58-3d508d468918 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1516 microversion: 2.1 time: 0.404809 Jul 20 14:50:20.408080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 208/416] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:19 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1516 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:20.504579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:20.515125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2031a9e6-e3ef-4a6a-acc0-de1ca3d6ac6a tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.361175 Jul 20 14:50:20.526126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 209/417] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:20 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:21.071168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b128fb66-0d2f-4e56-a8a1-a90e3e46b2c2 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Action: 'create', calling method: .version_select at 0x7f7b15df3670>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-878823478"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:21.414455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b128fb66-0d2f-4e56-a8a1-a90e3e46b2c2 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.495366 Jul 20 14:50:21.415123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 209/418] 10.222.0.15 () {60 vars in 1154 bytes} [Wed Jul 20 14:50:20 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 497 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 20 14:50:21.427708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-98e68bc4-1cfd-44cf-bf41-be0b0dfbef13 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b166c2550>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:21.495222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-98e68bc4-1cfd-44cf-bf41-be0b0dfbef13 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 200 len: 344 microversion: 2.1 time: 0.072617 Jul 20 14:50:21.495879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 210/419] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:50:21 2022] GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 344 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:21.531820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-b45ccbda-5fc6-455a-be97-d502c113fe59 req-4c1ee3b8-13c4-4f2f-a1c6-73ace51da733 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f3521d1-81cb-4217-b43a-b84b94911839", "tag": "ae0ff63e-d9ee-41cd-875c-523dfce9bd0c"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:21.532743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:21.533593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:21.540675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:21.541378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:21.542904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b45ccbda-5fc6-455a-be97-d502c113fe59 req-4c1ee3b8-13c4-4f2f-a1c6-73ace51da733 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:21.543251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:21.543692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b45ccbda-5fc6-455a-be97-d502c113fe59 req-4c1ee3b8-13c4-4f2f-a1c6-73ace51da733 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:21.544014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:21.559697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-b45ccbda-5fc6-455a-be97-d502c113fe59 req-4c1ee3b8-13c4-4f2f-a1c6-73ace51da733 service nova] Creating event network-changed:ae0ff63e-d9ee-41cd-875c-523dfce9bd0c for instance 2f3521d1-81cb-4217-b43a-b84b94911839 on hv-cinder-84902 Jul 20 14:50:21.565385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-b45ccbda-5fc6-455a-be97-d502c113fe59 req-4c1ee3b8-13c4-4f2f-a1c6-73ace51da733 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038072 Jul 20 14:50:21.566732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 210/420] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:50:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:21.763536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:21.772741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-945eca8d-c7e5-4c01-8708-5834740d9df1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.244984 Jul 20 14:50:21.784917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 211/421] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:21 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:50:22.036908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:22.037927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:22.045434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.046280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.047303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.047790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.077134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'flavor' on Instance uuid 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:22.169054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72ad4c7c-f55e-4fcc-bfa0-6b03135ac391 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/action" status: 200 len: 32 microversion: 2.1 time: 0.136546 Jul 20 14:50:22.169671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 211/422] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:50:22 2022] POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/action => generated 32 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 14:50:22.179987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.180597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:22.187108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.187797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.188870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.189539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.354956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:22.362971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-02504f3c-9a1a-4323-b82b-a1af6a982d99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.186859 Jul 20 14:50:22.364517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 212/423] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:50:22.512919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-deee9a54-321f-417a-942c-4793db86cf82 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e4b5e0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.566242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-deee9a54-321f-417a-942c-4793db86cf82 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 200 len: 345 microversion: 2.1 time: 0.060695 Jul 20 14:50:22.566840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 212/424] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 345 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:22.578029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1903580b-9c11-43c1-a0fa-aa2927798b62 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Action: 'create', calling method: .version_select at 0x7f7b1673dca0>, body: {"snapshot": {"volume_id": "b77d9054-378e-4920-b973-fdc320d807f4", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-51903510"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:22.756114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1903580b-9c11-43c1-a0fa-aa2927798b62 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "POST /compute/v2.1/os-snapshots" status: 200 len: 285 microversion: 2.1 time: 0.181886 Jul 20 14:50:22.762070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 213/425] 10.222.0.15 () {60 vars in 1161 bytes} [Wed Jul 20 14:50:22 2022] POST /compute/v2.1/os-snapshots => generated 285 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:22.767348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7bce1018-89e8-4abf-8ed6-2e7830ac8765 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15fdd0d0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.789887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.790592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:22.799282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.800000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.800859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:22.801517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:22.809273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7bce1018-89e8-4abf-8ed6-2e7830ac8765 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.046244 Jul 20 14:50:22.809851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 213/426] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:22.820030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c15121f9-99a8-4211-8037-3c526041813d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e3c940>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.857005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c15121f9-99a8-4211-8037-3c526041813d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots" status: 200 len: 288 microversion: 2.1 time: 0.041011 Jul 20 14:50:22.857574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 214/427] 10.222.0.15 () {58 vars in 1139 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/os-snapshots => generated 288 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:50:22.867853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0eb836c8-d1f1-4b86-95fb-49b16b179a7f tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e3c820>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:22.904141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0eb836c8-d1f1-4b86-95fb-49b16b179a7f tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.039776 Jul 20 14:50:22.904781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 215/428] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:22.980717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:22.988614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8e11bd42-be0d-4580-8bc4-1e53535c7d21 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.202554 Jul 20 14:50:22.989186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 214/429] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:22 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:23.381855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:23.382587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:23.389102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:23.389551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:23.390311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:23.390742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:23.529638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:23.537613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80567e07-4fd1-4d43-8605-134868fa7f7a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.159731 Jul 20 14:50:23.538102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 216/430] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:23 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:23.916250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-84ac557b-d102-4bb8-9978-7a4094990c58 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b16873c10>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:23.947387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-84ac557b-d102-4bb8-9978-7a4094990c58 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.034832 Jul 20 14:50:23.947950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 215/431] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:23 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:24.005685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:24.006296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:24.012900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:24.013349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:24.014111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:24.014546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:24.148808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:24.157194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f1d91dd-deb9-423d-864b-b7eb13fbd0cc tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.155536 Jul 20 14:50:24.157735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 217/432] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:24 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:24.555627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:24.556230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:24.563014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:24.563513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:24.564294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:24.564701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:24.699010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:24.706721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-10d393c0-96ff-4b6c-a2e1-8839d5de1aea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.155018 Jul 20 14:50:24.707321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 216/433] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:24 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:24.959038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4c225af7-eda9-4b64-920b-9a0703db3110 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15df3280>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:24.991320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4c225af7-eda9-4b64-920b-9a0703db3110 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.036193 Jul 20 14:50:24.991889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 218/434] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:24 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:25.175771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:25.176310 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:25.183590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:25.184044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:25.184820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:25.185239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:25.335074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:25.345187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-75d2c543-8563-4059-bc23-6d2b29e6c3e3 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.174613 Jul 20 14:50:25.345774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 217/435] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:25 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:25.724603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:25.725112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:25.731803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:25.732247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:25.733002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:25.733428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:25.865458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:25.873470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6618e8c9-57ca-4a46-8b67-2aa4c5eb2a74 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.153156 Jul 20 14:50:25.874175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 219/436] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:25 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:26.002664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c6335ac2-060a-44ca-970e-0e54c5ec0e8d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b16604700>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:26.032532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c6335ac2-060a-44ca-970e-0e54c5ec0e8d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.033681 Jul 20 14:50:26.033098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 218/437] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:25 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:26.364464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:26.365073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:26.371418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:26.371867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:26.372627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:26.373063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:26.514728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:26.522954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef631b0f-4a45-49da-9af0-5fef272c87ff tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.162396 Jul 20 14:50:26.523567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 220/438] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:26 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:26.890760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:26.891404 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:26.893304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:26.899008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:26.899482 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:26.900247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:26.900679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:27.041971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:27.044564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a70da4a5-3700-45ab-9727-98999aae1b92 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15df3700>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:27.050228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3de4fc91-b356-4ade-8141-58bd8a00c4b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.163949 Jul 20 14:50:27.050908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 219/439] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:26 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:27.075611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a70da4a5-3700-45ab-9727-98999aae1b92 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.034935 Jul 20 14:50:27.076203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 221/440] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:27 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:27.541042 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:27.541716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:27.549086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:27.549642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:27.550967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:27.551626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:27.708995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:27.718746 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fbd8fd14-24f9-4ab1-b4a4-8d37d537b886 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.181475 Jul 20 14:50:27.719412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 220/441] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:27 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:28.069234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:28.069868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:28.078953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:28.079423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:28.080488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:28.081246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:28.092444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-32d2fbc2-a871-44c3-8715-26b8641d9a9f tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b166044c0>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:28.129082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-32d2fbc2-a871-44c3-8715-26b8641d9a9f tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 286 microversion: 2.1 time: 0.043778 Jul 20 14:50:28.135693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 221/442] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:28 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 286 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:28.141884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4df0ebad-bf41-4fc0-8b26-e2864b7ab1b1 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b1a1e5b80>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:28.208069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4df0ebad-bf41-4fc0-8b26-e2864b7ab1b1 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "DELETE /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 202 len: 0 microversion: 2.1 time: 0.070850 Jul 20 14:50:28.208632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 222/443] 10.222.0.15 () {58 vars in 1253 bytes} [Wed Jul 20 14:50:28 2022] DELETE /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:50:28.222705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-49ab3256-1c02-4873-8e90-f836da9479af tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b165fc5e0>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:28.287748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-49ab3256-1c02-4873-8e90-f836da9479af tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 200 len: 285 microversion: 2.1 time: 0.072233 Jul 20 14:50:28.288685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 223/444] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:28 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 285 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:28.701420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:28.710514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5425b767-2724-4aef-b9aa-883d71df5510 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.645123 Jul 20 14:50:28.711403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 222/445] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:28 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:28.737234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:28.737890 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:28.748417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:28.749029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:28.750098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:28.750695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:28.940077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:28.948186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c633a196-0d1e-4790-a352-53df08a5b69d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.215052 Jul 20 14:50:28.948799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 224/446] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:28 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:29.301048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7086b2a1-6985-4b6e-924b-2d30c894339d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15df3790>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:29.327243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-7086b2a1-6985-4b6e-924b-2d30c894339d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] HTTP exception thrown: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7086b2a1-6985-4b6e-924b-2d30c894339d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Request method failure captured: Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 HTTP/1.1 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BYsCtyohwR4vWCP-ro3y5lrAakIpWgnKM5iwTxVcci_6V9lxPNdwRJ1_c3qic9RnL8MFST0kOjnEAvDJ9OpktRb4kzw-wWAUac__VoseqnbfDtTS8M_9V0-QvwlkCCGjKe952OzmsaTYcIStqNReqqKiYHJjsu0ZdDLVnjV9Op54k8 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:29.329565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_f0af7082b1c04ea687097e35e9050807"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}] Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 54f0a23979014886bfed56cf62a0fa70 Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4eeccf51-4a04-49aa-834b-c8acf6734b34'} Jul 20 14:50:29.331090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. (HTTP 404) (Request-ID: req-c5b1ac85-28be-4e94-a726-7cadc7d927dd) Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.332125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:29.333499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 20 14:50:29.335122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7086b2a1-6985-4b6e-924b-2d30c894339d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Returning 404 to user: Snapshot 4eeccf51-4a04-49aa-834b-c8acf6734b34 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:29.336952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7086b2a1-6985-4b6e-924b-2d30c894339d tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34" status: 404 len: 111 microversion: 2.1 time: 0.034314 Jul 20 14:50:29.337948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 223/447] 10.222.0.15 () {58 vars in 1250 bytes} [Wed Jul 20 14:50:29 2022] GET /compute/v2.1/os-snapshots/4eeccf51-4a04-49aa-834b-c8acf6734b34 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:29.340676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6af2a581-fc30-4687-8a45-02624b3943eb tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b165fc430>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:29.407957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6af2a581-fc30-4687-8a45-02624b3943eb tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 202 len: 0 microversion: 2.1 time: 0.071270 Jul 20 14:50:29.408601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 225/448] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:50:29 2022] DELETE /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 0 bytes in 73 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:50:29.417075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e6248cd2-6bfd-47ff-813e-56b5b8a6201e tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e4b040>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:29.468664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e6248cd2-6bfd-47ff-813e-56b5b8a6201e tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 200 len: 344 microversion: 2.1 time: 0.055565 Jul 20 14:50:29.469238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 224/449] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:50:29 2022] GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 344 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:29.731104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:29.731104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:29.735572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:29.736283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:29.737216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:29.737998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:29.893066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:29.902671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-09e10654-2662-4619-937b-68c3fe022132 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.178566 Jul 20 14:50:29.903314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 226/450] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:29 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:29.967766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:29.968397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:29.975072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:29.975529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:29.976360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:29.976868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:30.135000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:30.144653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f5b37558-aea5-4176-8894-59a75b2a0005 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.183038 Jul 20 14:50:30.145206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 225/451] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:29 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:30.164173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-71164946-6e01-4840-8c0e-449a013462a0 req-af7d16dd-0a0e-458b-93bf-b0a09ea59fdb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38c6f67e-423e-44d8-abcb-73298f72167b", "name": "network-vif-plugged", "status": "completed", "tag": "9e75b368-eeb6-4a49-897c-6366158a4049"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:30.173273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-71164946-6e01-4840-8c0e-449a013462a0 req-af7d16dd-0a0e-458b-93bf-b0a09ea59fdb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:30.173743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-71164946-6e01-4840-8c0e-449a013462a0 req-af7d16dd-0a0e-458b-93bf-b0a09ea59fdb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:30.187934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-71164946-6e01-4840-8c0e-449a013462a0 req-af7d16dd-0a0e-458b-93bf-b0a09ea59fdb service nova] Creating event network-vif-plugged:9e75b368-eeb6-4a49-897c-6366158a4049 for instance 38c6f67e-423e-44d8-abcb-73298f72167b on hv-cinder-84902 Jul 20 14:50:30.193029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-71164946-6e01-4840-8c0e-449a013462a0 req-af7d16dd-0a0e-458b-93bf-b0a09ea59fdb service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032906 Jul 20 14:50:30.193647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 227/452] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:50:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:50:30.481743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17200bf-5c32-447f-8f83-5151ba5c9193 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e4b3a0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:30.530197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-d17200bf-5c32-447f-8f83-5151ba5c9193 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] HTTP exception thrown: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17200bf-5c32-447f-8f83-5151ba5c9193 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Request method failure captured: Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 HTTP/1.1 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BYsCtyohwR4vWCP-ro3y5lrAakIpWgnKM5iwTxVcci_6V9lxPNdwRJ1_c3qic9RnL8MFST0kOjnEAvDJ9OpktRb4kzw-wWAUac__VoseqnbfDtTS8M_9V0-QvwlkCCGjKe952OzmsaTYcIStqNReqqKiYHJjsu0ZdDLVnjV9Op54k8 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:30.532286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_f0af7082b1c04ea687097e35e9050807"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}] Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 54f0a23979014886bfed56cf62a0fa70 Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'b77d9054-378e-4920-b973-fdc320d807f4'} Jul 20 14:50:30.533760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. (HTTP 404) (Request-ID: req-e9cc12c4-c8e4-41c1-8af9-608e18005152) Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.534724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:30.536236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 20 14:50:30.537480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17200bf-5c32-447f-8f83-5151ba5c9193 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Returning 404 to user: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:30.538712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d17200bf-5c32-447f-8f83-5151ba5c9193 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 404 len: 109 microversion: 2.1 time: 0.056129 Jul 20 14:50:30.543363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 226/453] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:50:30 2022] GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 109 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:30.554861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8802c8ad-1233-4307-b17e-dcbb9c60faf4 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b168983a0>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:30.585760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-8802c8ad-1233-4307-b17e-dcbb9c60faf4 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] HTTP exception thrown: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8802c8ad-1233-4307-b17e-dcbb9c60faf4 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Request method failure captured: Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 HTTP/1.1 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BYsCtyohwR4vWCP-ro3y5lrAakIpWgnKM5iwTxVcci_6V9lxPNdwRJ1_c3qic9RnL8MFST0kOjnEAvDJ9OpktRb4kzw-wWAUac__VoseqnbfDtTS8M_9V0-QvwlkCCGjKe952OzmsaTYcIStqNReqqKiYHJjsu0ZdDLVnjV9Op54k8 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:50:30.587850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_f0af7082b1c04ea687097e35e9050807"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}] Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 54f0a23979014886bfed56cf62a0fa70 Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': 'b77d9054-378e-4920-b973-fdc320d807f4'} Jul 20 14:50:30.589328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. (HTTP 404) (Request-ID: req-782a5fc2-b52d-4181-afa5-2c5798edff5a) Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.590306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 20 14:50:30.591711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 20 14:50:30.593054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8802c8ad-1233-4307-b17e-dcbb9c60faf4 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Returning 404 to user: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8802c8ad-1233-4307-b17e-dcbb9c60faf4 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 404 len: 109 microversion: 2.1 time: 0.039158 Jul 20 14:50:30.594341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 228/454] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:50:30 2022] DELETE /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:30.598990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e24bb89-33e4-41b8-b319-a2f9d79bf828 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Calling method '.version_select at 0x7f7b15e4bdc0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:30.629270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-0e24bb89-33e4-41b8-b319-a2f9d79bf828 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] HTTP exception thrown: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e24bb89-33e4-41b8-b319-a2f9d79bf828 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Request method failure captured: Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 HTTP/1.1 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BYsCtyohwR4vWCP-ro3y5lrAakIpWgnKM5iwTxVcci_6V9lxPNdwRJ1_c3qic9RnL8MFST0kOjnEAvDJ9OpktRb4kzw-wWAUac__VoseqnbfDtTS8M_9V0-QvwlkCCGjKe952OzmsaTYcIStqNReqqKiYHJjsu0ZdDLVnjV9Op54k8 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:30.631242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_f0af7082b1c04ea687097e35e9050807"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/f0af7082b1c04ea687097e35e9050807"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/f0af7082b1c04ea687097e35e9050807"}]}] Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: f0af7082b1c04ea687097e35e9050807 Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1124681074 Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 54f0a23979014886bfed56cf62a0fa70 Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1124681074-project Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'b77d9054-378e-4920-b973-fdc320d807f4'} Jul 20 14:50:30.632826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. (HTTP 404) (Request-ID: req-676a26eb-219a-49ba-8cfc-3ee55c42677a) Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.633618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 14:50:30.634658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 20 14:50:30.635821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e24bb89-33e4-41b8-b319-a2f9d79bf828 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] Returning 404 to user: Volume b77d9054-378e-4920-b973-fdc320d807f4 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e24bb89-33e4-41b8-b319-a2f9d79bf828 tempest-VolumesSnapshotsTestJSON-1124681074 tempest-VolumesSnapshotsTestJSON-1124681074-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4" status: 404 len: 109 microversion: 2.1 time: 0.037674 Jul 20 14:50:30.637331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 227/455] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 14:50:30 2022] GET /compute/v2.1/os-volumes/b77d9054-378e-4920-b973-fdc320d807f4 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:30.921735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:30.922446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:30.930179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:30.930627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:30.931442 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:30.932000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:30.936346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:31.076864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:31.085076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0da0423b-3b82-42d9-9c25-65047e1f1c6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.168118 Jul 20 14:50:31.085658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 229/456] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:30 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:31.162690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:31.163333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:31.175975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:31.176355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:31.177156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:31.177599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:31.369151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:31.377229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4cb1c7d-3353-40e4-b35c-0307a65a4fe5 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.218730 Jul 20 14:50:31.377832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 228/457] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:31 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:32.102266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:32.102908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:32.109893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:32.110343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:32.111267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:32.111680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:32.281735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:32.290720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1c5f10b0-9f56-4973-8ee6-887e3a7990cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.192378 Jul 20 14:50:32.291311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 230/458] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:32 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:32.394115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:32.395502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:32.401708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:32.402361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:32.403343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:32.404241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:32.571184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:32.582552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b485c8af-c85d-465a-94b7-81d0f85f690d tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1314 microversion: 2.1 time: 0.192180 Jul 20 14:50:32.583408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 229/459] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:32 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1314 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:33.315138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:33.316349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:33.323654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.324116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.324859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.325453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.478082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:33.487461 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b993c053-4e0a-4a42-834c-8fe4978ecca2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.177050 Jul 20 14:50:33.488336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 231/460] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:33 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:33.600851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:33.601485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:33.608091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.608612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.609484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.609976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.764329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:33.772538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-03ce97fb-8e97-4bae-ad72-bcab08df382e tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1464 microversion: 2.1 time: 0.176741 Jul 20 14:50:33.784825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 230/461] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:33 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1464 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:33.817554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:33.818127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:33.824240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.824760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.825639 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.826167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.851126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:50:33.937780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-23ec23e8-090d-4ef1-8d86-68326b0e69c1 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "DELETE /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 204 len: 0 microversion: 2.1 time: 0.123532 Jul 20 14:50:33.938672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 232/462] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:50:33 2022] DELETE /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:50:33.947347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:33.948067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:33.955882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.956329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:33.957186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:33.957608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:34.091635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:34.100328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4e565a1d-9e7e-4791-92f9-ea525c7e6d86 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1470 microversion: 2.1 time: 0.157029 Jul 20 14:50:34.100919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 231/463] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:33 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1470 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:34.506156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:34.506911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:34.513286 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:34.513717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:34.514480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:34.514902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:34.664716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:34.672389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-90befd9b-a1ed-46e0-bbc1-a84a9827111b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.170987 Jul 20 14:50:34.679092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 233/464] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:34 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:35.121367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:35.123322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:35.130369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:35.131086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:35.131875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:35.132309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:35.282932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:35.292181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea97fd54-e8fe-40cd-a307-0da26aa2e087 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1470 microversion: 2.1 time: 0.177705 Jul 20 14:50:35.303337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 232/465] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:35 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1470 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:35.691452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:35.692038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:35.701923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:35.702489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:35.703341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:35.703854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:35.840224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:35.848285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-620ebedd-244d-4ed3-b4ef-d65c67c2bb19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.160980 Jul 20 14:50:35.848923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 234/466] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:35 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:36.317015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:36.317727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:36.319772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=70 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:36.325504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:36.325968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:36.326780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:36.327449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:36.521118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:36.533058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61d0135c-543c-4d2f-a546-14259484843b tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1470 microversion: 2.1 time: 0.219825 Jul 20 14:50:36.533730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 233/467] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:36 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1470 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:36.870147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:36.877530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:36.892748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:36.893232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:36.894055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:36.894494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:37.079368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-8c238027-1c28-4874-ae66-06f8a9a0e50b req-7852b923-a98c-4587-ae35-ea05b4da6ecf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38c6f67e-423e-44d8-abcb-73298f72167b", "name": "network-vif-deleted", "tag": "9e75b368-eeb6-4a49-897c-6366158a4049"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:37.104505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-8c238027-1c28-4874-ae66-06f8a9a0e50b req-7852b923-a98c-4587-ae35-ea05b4da6ecf service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:37.105245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-8c238027-1c28-4874-ae66-06f8a9a0e50b req-7852b923-a98c-4587-ae35-ea05b4da6ecf service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:37.145466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-8c238027-1c28-4874-ae66-06f8a9a0e50b req-7852b923-a98c-4587-ae35-ea05b4da6ecf service nova] Creating event network-vif-deleted:9e75b368-eeb6-4a49-897c-6366158a4049 for instance 38c6f67e-423e-44d8-abcb-73298f72167b on hv-cinder-84902 Jul 20 14:50:37.150991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-8c238027-1c28-4874-ae66-06f8a9a0e50b req-7852b923-a98c-4587-ae35-ea05b4da6ecf service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075689 Jul 20 14:50:37.151915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 234/468] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:50:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:50:37.181871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:37.190476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e0ac7f40-8399-4420-a554-2214434ca43b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.327548 Jul 20 14:50:37.191191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 235/469] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:36 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:37.551269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:37.551973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:37.559827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:37.560407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:37.561393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:37.562182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:37.599390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:37.615639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:37.616087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:37.616538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lazy-loading 'fault' on Instance uuid 38c6f67e-423e-44d8-abcb-73298f72167b {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:50:37.674792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1750ad6625e4cc0b430a81ed188e538', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '50a5f7380c804eba98ab4a8ee65dc1d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:37.682929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32468596-3252-427e-9641-7f98fd918db0 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 200 len: 1304 microversion: 2.1 time: 0.136655 Jul 20 14:50:37.683530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 235/470] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:37 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 1304 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:38.218617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:38.219634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:38.231264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:38.231705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:38.232476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:38.232931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:38.378376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:38.386569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3ac3e576-23be-445d-a98f-2942d0ee8ee7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.182687 Jul 20 14:50:38.387162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 236/471] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:38 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:38.700305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:38.700925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] [instance: 38c6f67e-423e-44d8-abcb-73298f72167b] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:38.707374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:38.707816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:38.708578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:38.709004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:38.723382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] HTTP exception thrown: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Request method failure captured: Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b HTTP/1.1 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BYUjRhqmPg6231kRG1NFbnehwuTZ0sXhE6cf56nWxJUdaYo1cDcRxkwgVVUtbbbrZnrieRwgzGh2GSkznS6AZk7rRbmPgA1igJWbpo98dEU31U8gHIYxpTO_4q-PSlZykFAVQyByNVXRI5ipndgGXvXH_DNZLNUvq-dtvgSSjaAsxM Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 50a5f7380c804eba98ab4a8ee65dc1d3 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesNegativeTest-1741408274 Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:50:38.725195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_50a5f7380c804eba98ab4a8ee65dc1d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/50a5f7380c804eba98ab4a8ee65dc1d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/50a5f7380c804eba98ab4a8ee65dc1d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/50a5f7380c804eba98ab4a8ee65dc1d3"}]}] Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesNegativeTest-1741408274 Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 50a5f7380c804eba98ab4a8ee65dc1d3 Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesNegativeTest-1741408274 Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesNegativeTest-1741408274-project Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e1750ad6625e4cc0b430a81ed188e538 Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesNegativeTest-1741408274-project Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. Jul 20 14:50:38.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '38c6f67e-423e-44d8-abcb-73298f72167b'} Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:38.727765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. Jul 20 14:50:38.729805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:50:38.731505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] Returning 404 to user: Instance 38c6f67e-423e-44d8-abcb-73298f72167b could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:38.731505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-344ddd85-82a2-4e5f-b1e4-2e50d12a3c19 tempest-VolumesNegativeTest-1741408274 tempest-VolumesNegativeTest-1741408274-project] 10.222.0.15 "GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b" status: 404 len: 111 microversion: 2.1 time: 0.030604 Jul 20 14:50:38.731505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 236/472] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:38 2022] GET /compute/v2.1/servers/38c6f67e-423e-44d8-abcb-73298f72167b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:50:39.406428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:39.407535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:39.417389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:39.417981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:39.419593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:39.420265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:39.572312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:39.580615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dc9ee1c6-9e70-45ec-8e95-30a6b62ce56e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.180619 Jul 20 14:50:39.581268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 237/473] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:39 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:40.601758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:40.602883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:40.614318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:40.614952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:40.616346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:40.617135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:40.783697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:40.792362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1fddaf6-36e4-46ac-80e5-26d57fa88f91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.197428 Jul 20 14:50:40.793243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 237/474] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:40 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:41.813274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:41.813854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:41.823988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:41.824589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:41.825557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:41.826158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:41.960354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:41.969200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-77298446-712e-44b3-9e27-0c0e41b868a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.162308 Jul 20 14:50:41.969786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 238/475] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:41 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:42.987297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:42.987928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:42.994423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:42.994877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:42.995689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:42.996100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:43.134342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:43.142461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bf3244e0-86c9-497b-b945-eae7cb21f66e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.159077 Jul 20 14:50:43.143035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 238/476] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:42 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:44.160504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:44.161054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:44.168206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:44.168719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:44.169670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:44.170361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:44.313506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:44.321782 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-165bbd60-4a01-40ae-8697-c6c29d5e4a30 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.165615 Jul 20 14:50:44.322351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 239/477] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:44 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:45.338994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:45.339629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:45.346194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:45.346649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:45.347436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:45.347892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:45.480495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:45.489383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-651f2535-2e27-4603-b3eb-ec3538620387 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.154239 Jul 20 14:50:45.489972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 239/478] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:45 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:46.506675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:46.507292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:46.514331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:46.514778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:46.515585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:46.516020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:46.651674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:46.659772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1cd2fff5-c4dc-4095-82e8-0cd3d2498c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.156964 Jul 20 14:50:46.660348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 240/479] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:46 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:50:47.677416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:47.678031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:47.684767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:47.685210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:47.685978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:47.686504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:47.850522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:47.862801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-687a96c2-a97b-4770-adc8-1a38c286d69c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.189414 Jul 20 14:50:47.863814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 240/480] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:47 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:48.880546 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:48.881140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:48.888029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:48.888477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:48.889254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:48.889675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:49.019684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:49.031732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ec2b462c-7d70-421c-a3f2-40a1cb5b16b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.154818 Jul 20 14:50:49.032621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 241/481] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:48 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:50.055569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:50.056522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:50.065816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.066524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.067745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.068440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.231617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:50.243506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d87b85d5-f469-418a-b63b-2c8a5346a32c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1502 microversion: 2.1 time: 0.189424 Jul 20 14:50:50.247210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 241/482] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:50 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1502 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:50:50.260253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5f2d29b1-4edc-49e9-9587-0a018644781e"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:50.261383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:50.268049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.268502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.269280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.269685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.293015 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Cannot 'attach_volume' instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 while it is in vm_state rescued Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/os-volume_attachments HTTP/1.1 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 74 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:50:50.294804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {"volumeAttachment": {"volumeId": "5f2d29b1-4edc-49e9-9587-0a018644781e"}} Jul 20 14:50:50.296408 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Cannot 'attach_volume' instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 while it is in vm_state rescued Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '304148dc-eb2a-4dec-a9ec-1cff5c5114b7', 'body': {'volumeAttachment': {'volumeId': '5f2d29b1-4edc-49e9-9587-0a018644781e'}}} Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:50:50.297217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 while it is in vm_state rescued Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:50:50.298621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 while it is in vm_state rescued Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 409 to user: Cannot 'attach_volume' instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 while it is in vm_state rescued {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9969d739-d630-4615-b2bd-d97c45a62805 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.040135 Jul 20 14:50:50.300062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 242/483] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:50:50 2022] POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/os-volume_attachments => generated 152 bytes in 41 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 20 14:50:50.306595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:50.307263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:50.313982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.314639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.315561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.316135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.382188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f90caee-3a92-4a93-8a83-5a4879afa928 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/action" status: 202 len: 0 microversion: 2.1 time: 0.079670 Jul 20 14:50:50.382816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 242/484] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:50:50 2022] POST /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7/action => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:50:50.390997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:50.391647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:50.398424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.398961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.399799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:50.400226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:50.536810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:50.553526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2799896f-2091-4f69-a63c-3c183cea0265 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1510 microversion: 2.1 time: 0.166242 Jul 20 14:50:50.554021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 243/485] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:50 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1510 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:51.572583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:51.573314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:51.580113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:51.580563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:51.581346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:51.581968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:51.716183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:51.723860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92935f87-d9f9-4b68-ae00-73e9930aa6e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1510 microversion: 2.1 time: 0.155702 Jul 20 14:50:51.724405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 243/486] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:51 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1510 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:52.742014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:52.742593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:52.748567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:52.749004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:52.749798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:52.750235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:52.770864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:52.897626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:52.907095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3d075826-b619-4fbe-88b3-55a26b320e4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1510 microversion: 2.1 time: 0.169283 Jul 20 14:50:52.907890 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 244/487] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:52 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1510 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:53.925361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:53.925958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:53.933889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:53.934657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:53.936075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:53.936824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:54.078213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:54.086050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c2f015c2-7f28-44d6-a27a-dceef586a3a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1510 microversion: 2.1 time: 0.164787 Jul 20 14:50:54.086636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 244/488] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:53 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1510 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:55.104381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:55.104983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:55.111923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:55.112361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:55.113147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:55.113585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:55.252620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:55.260645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-77e624ad-92ff-4004-9416-43bb60cabeba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1510 microversion: 2.1 time: 0.160316 Jul 20 14:50:55.261207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 245/489] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:55 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1510 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:56.278298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:50:56.279083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:50:56.285530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:56.285969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:56.286722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:50:56.287183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:50:56.425957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:50:56.433678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5592fb42-f0c8-4144-a70f-3d42503c9267 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1516 microversion: 2.1 time: 0.159732 Jul 20 14:50:56.434225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 245/490] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:50:56 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1516 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:50:58.847452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-da6ab718-80ff-4457-bda1-e9eef18092e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: .version_select at 0x7f7b16604c10>, body: {"keypair": {"name": "tempest-keypair-509396105", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKXMCjeXdYDa5Blj/jKEjbfjH1zf4ZzUUz8lsryzs5wPZwTm+ONkCNxRBHR7IIJrh/Z6XvDK8mJ5KOh6ZYIYtLjbheOh9h6cEcyrSj3Humjy9MVOxbptVTxxRUtbJ6GZcA=="}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:50:58.853024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:50:58.866355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-da6ab718-80ff-4457-bda1-e9eef18092e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'key_pairs'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:50:58.870443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-da6ab718-80ff-4457-bda1-e9eef18092e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for user 938083ff74d84224be142a26d8e17ad6 and project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'key_pairs'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:50:58.899519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-da6ab718-80ff-4457-bda1-e9eef18092e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.056475 Jul 20 14:50:58.900128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 246/491] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:50:58 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:00.335264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1371000206", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "ce763c04-6208-4594-840c-20615d6164a4"}], "security_groups": [{"name": "tempest-securitygroup--2141250174"}], "key_name": "tempest-keypair-509396105", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:00.467596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] validate_networks() for [('ce763c04-6208-4594-840c-20615d6164a4', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:51:00.966296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:51:00.966704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:51:00.967190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:51:00.967800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:51:00.968213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:51:00.985245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:00.985845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:00.986771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:00.987295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:01.006450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:51:01.010926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Getting quotas for user 938083ff74d84224be142a26d8e17ad6 and project 5c6ce84a298f4daa938864ec78d7a996. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:51:01.023601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:51:01.025984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:51:01.074274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be1ed956-40ba-4a48-a991-6df8f4071a66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.743607 Jul 20 14:51:01.080025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 246/492] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:51:00 2022] POST /compute/v2.1/servers => generated 409 bytes in 745 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:51:01.085771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd6abdff-3166-4361-b6aa-8ccbd0f8a176 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:01.086389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd6abdff-3166-4361-b6aa-8ccbd0f8a176 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:01.145725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd6abdff-3166-4361-b6aa-8ccbd0f8a176 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:01.155429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd6abdff-3166-4361-b6aa-8ccbd0f8a176 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1245 microversion: 2.1 time: 0.074000 Jul 20 14:51:01.156091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 247/493] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1245 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:51:02.173656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:02.174252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:02.180554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:02.180994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:02.181755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:02.182184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:02.263004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:02.271808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba918683-f132-40d8-9c2f-b81697dccb8a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1249 microversion: 2.1 time: 0.102104 Jul 20 14:51:02.272355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 247/494] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:02 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1249 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:03.289501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:03.290280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:03.297193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:03.297580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:03.298475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:03.298920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:03.362790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:03.370474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-98b328d1-6012-4192-a22c-371347a8d602 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1249 microversion: 2.1 time: 0.084880 Jul 20 14:51:03.371066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 248/495] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1249 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:04.387435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:04.388090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:04.394415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:04.394872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:04.395681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:04.396119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:04.469015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:04.482713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d49ec39-2176-4834-9e13-b31188f35ca8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1241 microversion: 2.1 time: 0.099045 Jul 20 14:51:04.484092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 248/496] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1241 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:05.502665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:05.503479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:05.511339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:05.511777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:05.512565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:05.512990 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:05.578077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:05.585865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8013dc49-38ad-4dca-9565-5d1c136d7d41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1297 microversion: 2.1 time: 0.088393 Jul 20 14:51:05.586407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 249/497] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1297 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:06.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:06.603609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:06.605658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:06.611374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:06.611847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:06.612601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:06.613035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:06.680784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:06.690957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86553b1a-996b-4f9b-8324-d2735d97a2a3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1303 microversion: 2.1 time: 0.091874 Jul 20 14:51:06.691596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 249/498] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1303 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:07.708565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:07.709127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:07.716087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:07.716536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:07.717314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:07.717739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:07.937577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:07.946917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-23b29642-22a3-4516-9cd4-48379ec189c4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.242328 Jul 20 14:51:07.947550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 250/499] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:08.077466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-eaa2558a-a260-470a-b911-37fde13ed8c8 req-b5425a75-a7c2-4b19-b8cc-7b4211cce019 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "75115162-3a7f-4a94-a703-190f8fdbc4c8", "tag": "94ca1675-607f-45ff-a048-c7df5f9c493a"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:08.092146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-eaa2558a-a260-470a-b911-37fde13ed8c8 req-b5425a75-a7c2-4b19-b8cc-7b4211cce019 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:08.099089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-eaa2558a-a260-470a-b911-37fde13ed8c8 req-b5425a75-a7c2-4b19-b8cc-7b4211cce019 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:08.109354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:08.119179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-eaa2558a-a260-470a-b911-37fde13ed8c8 req-b5425a75-a7c2-4b19-b8cc-7b4211cce019 service nova] Creating event network-changed:94ca1675-607f-45ff-a048-c7df5f9c493a for instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 on hv-cinder-84902 Jul 20 14:51:08.124846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-eaa2558a-a260-470a-b911-37fde13ed8c8 req-b5425a75-a7c2-4b19-b8cc-7b4211cce019 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052126 Jul 20 14:51:08.125692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 250/500] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:08.964262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:08.964848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:08.971856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:08.972321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:08.973176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:08.973602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:09.106972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:09.115479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b90d5839-744d-42b1-8783-3054fcdade62 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.155111 Jul 20 14:51:09.116050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 251/501] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:10.133984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:10.134674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:10.141923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:10.142412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:10.143204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:10.143640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:10.285561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:10.293316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5086d052-7f0c-4dfe-976c-16f23db5d202 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.164302 Jul 20 14:51:10.293884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 251/502] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:10.322757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1992886051", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "networks": [{"uuid": "4f187311-3105-4dca-b98e-c91745d78e9f"}]}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:10.382684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.network.neutron [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] validate_networks() for [('4f187311-3105-4dca-b98e-c91745d78e9f', None, None, None, None, None)] {{(pid=124397) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:51:10.769612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Flavor limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:51:10.770035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Image limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:51:10.770504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Flavor pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:51:10.770907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Image pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:51:10.771362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:51:10.781035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:10.781478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:10.782262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:10.782695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:10.799727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Getting quotas for project 114da792d3f54f38b215aaeea69fd454. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:51:10.803861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Getting quotas for user c29795aae948425b87a6852b41a80c5d and project 114da792d3f54f38b215aaeea69fd454. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:51:10.816015 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Going to run 1 instances... {{(pid=124397) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:51:10.817532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124397) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:51:10.859812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR oslo.messaging._drivers.impl_rabbit [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [5601567a-37b1-4936-a184-4983e01cfcb1] AMQP server on 10.222.0.15:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 20 14:51:11.310072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:11.310892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:11.312911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:11.313579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:11.319600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:11.320058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:11.320858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:11.321294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:11.484731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:11.490655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cf68090b-84b4-4c7c-bdaf-8c54b23fae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.184289 Jul 20 14:51:11.491890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 252/503] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:11.868332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [5601567a-37b1-4936-a184-4983e01cfcb1] Reconnected to AMQP server on 10.222.0.15:5672 via [amqp] client with port 45004. Jul 20 14:51:11.873853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-23e87e13-3e01-4519-9610-9a098d53d908 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.555439 Jul 20 14:51:11.874526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 252/504] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:51:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 1557 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:51:11.885359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ce88e4b-6a01-45f9-8894-18b7136e70f9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:11.886151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ce88e4b-6a01-45f9-8894-18b7136e70f9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:11.943386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ce88e4b-6a01-45f9-8894-18b7136e70f9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:11.950283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ce88e4b-6a01-45f9-8894-18b7136e70f9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1214 microversion: 2.1 time: 0.069146 Jul 20 14:51:11.951218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 253/505] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:11 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1214 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:12.511154 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:12.511781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:12.518831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:12.519501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:12.520719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:12.521214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:12.690564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:12.699694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d2a3af97-f1df-4c70-8801-c45fdb55858d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.193567 Jul 20 14:51:12.700311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 253/506] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:12.967255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:12.968199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:12.974757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:12.975379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:12.976197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:12.976731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:13.073535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:13.081771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c6d50eba-8a6e-4d0a-b10f-679bc03214c6 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1210 microversion: 2.1 time: 0.118423 Jul 20 14:51:13.082339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 254/507] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:12 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1210 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:13.719685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:13.720888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:13.744732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:13.745208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:13.757061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:13.757061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:13.911651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:13.923435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dd524254-9ada-43dd-b63f-970a91bbd52d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.208883 Jul 20 14:51:13.924849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 254/508] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:14.100388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:14.101863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:14.108865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:14.109642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:14.110974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:14.111619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:14.187311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:14.196727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b8219c68-5f67-4a3b-8993-b2abacfb1e5d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1272 microversion: 2.1 time: 0.101220 Jul 20 14:51:14.197409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 255/509] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:14 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1272 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:14.943034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:14.943694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:14.951144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:14.951598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:14.952368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:14.952801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:15.099729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:15.109027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-70442377-44c1-47aa-a843-fd4d3bd86e2c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.171174 Jul 20 14:51:15.109612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 255/510] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:15.217749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:15.218247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:15.225077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:15.225716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:15.226565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:15.227031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:15.283534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-71c92683-df87-4e41-89e3-992a687c1089 req-9c498ad2-dab3-42dd-8577-acb2daee8cc3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "192968a0-0654-43f6-8846-3e1bbb536fe5", "tag": "dd51169e-9f1a-4269-9378-37fbd6dec56a"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:15.294791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-71c92683-df87-4e41-89e3-992a687c1089 req-9c498ad2-dab3-42dd-8577-acb2daee8cc3 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:15.295407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-71c92683-df87-4e41-89e3-992a687c1089 req-9c498ad2-dab3-42dd-8577-acb2daee8cc3 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:15.308353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-71c92683-df87-4e41-89e3-992a687c1089 req-9c498ad2-dab3-42dd-8577-acb2daee8cc3 service nova] Creating event network-changed:dd51169e-9f1a-4269-9378-37fbd6dec56a for instance 192968a0-0654-43f6-8846-3e1bbb536fe5 on hv-cinder-84902 Jul 20 14:51:15.313445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-71c92683-df87-4e41-89e3-992a687c1089 req-9c498ad2-dab3-42dd-8577-acb2daee8cc3 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034638 Jul 20 14:51:15.314050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 256/511] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:51:15.377237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:15.385588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-11233a6b-6448-43fa-ac20-c664b893ae86 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.172916 Jul 20 14:51:15.386191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 256/512] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:15 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:16.128074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:16.128767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:16.136302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:16.136748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:16.137516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:16.137980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:16.286561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:16.294714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cc208195-d60d-473c-9049-bc0a341ed0bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.171538 Jul 20 14:51:16.295317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 257/513] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:16.407015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:16.407696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:16.414338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:16.414781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:16.415587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:16.416017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:16.553033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:16.560757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c94eadfe-7f3c-4830-90e4-d1d791795237 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.158302 Jul 20 14:51:16.561363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 257/514] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:16 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:17.314620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:17.315498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:17.322474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:17.323111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:17.324420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:17.325133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:17.479961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:17.488152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1ba2eb9-e3ef-4c9f-9ec7-ed20ef3bf239 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.179884 Jul 20 14:51:17.488804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 258/515] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:17 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:17.581076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:17.582402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:17.588410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:17.588854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:17.589637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:17.590057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:17.728726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:17.740227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d34ba89-11cd-44a9-b97d-55fb90c63d4b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.164090 Jul 20 14:51:17.740820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 258/516] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:17 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:18.506241 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:18.506854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:18.513965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:18.514420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:18.515274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:18.515712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:18.672689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:18.688915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1495c2a2-39b8-41c8-80e3-1b888960c60c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.186665 Jul 20 14:51:18.689506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 259/517] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:18.760143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:18.760779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:18.767496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:18.768130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:18.768942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:18.769584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:18.912559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:18.920827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c24e260-8144-4e35-9b46-2fc8dec1e26d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.164959 Jul 20 14:51:18.921410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 259/518] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:18 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:19.705808 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:19.706454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:19.713697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:19.714148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:19.714922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:19.715411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:19.875542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:19.883605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3abb6a79-ebc6-4913-bbfc-37b03170a8de tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.181810 Jul 20 14:51:19.884181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 260/519] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:19.940417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:19.941030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:19.947583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:19.948034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:19.948796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:19.949224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:20.100075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:20.108111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-01cdfcf9-6397-4859-8740-999680a2e957 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.171951 Jul 20 14:51:20.108689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 260/520] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:19 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:20.901114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:20.901670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:20.904692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:20.905372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=12 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:20.911684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:20.912135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:20.913021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:20.913409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:21.048590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:21.057407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f678f70f-4163-4416-b86c-58cb48f8f233 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.160542 Jul 20 14:51:21.057995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 261/521] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:21.125176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:21.125770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:21.132462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:21.133193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:21.133940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:21.134363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:21.286652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:21.296862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3f0988a3-6017-4d11-91a8-87a2875d8038 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.175460 Jul 20 14:51:21.297716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 261/522] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:21 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:21.437202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-cfad3389-2f0b-44af-bdb2-4fd8789f0565 req-3d40c36c-3ce0-437c-beab-87890795ae71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75115162-3a7f-4a94-a703-190f8fdbc4c8", "name": "network-vif-plugged", "status": "completed", "tag": "94ca1675-607f-45ff-a048-c7df5f9c493a"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:21.446212 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-cfad3389-2f0b-44af-bdb2-4fd8789f0565 req-3d40c36c-3ce0-437c-beab-87890795ae71 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:21.446728 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-cfad3389-2f0b-44af-bdb2-4fd8789f0565 req-3d40c36c-3ce0-437c-beab-87890795ae71 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:21.460190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-cfad3389-2f0b-44af-bdb2-4fd8789f0565 req-3d40c36c-3ce0-437c-beab-87890795ae71 service nova] Creating event network-vif-plugged:94ca1675-607f-45ff-a048-c7df5f9c493a for instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 on hv-cinder-84902 Jul 20 14:51:21.466022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-cfad3389-2f0b-44af-bdb2-4fd8789f0565 req-3d40c36c-3ce0-437c-beab-87890795ae71 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032536 Jul 20 14:51:21.466699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 262/523] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:22.077521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:22.078460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:22.080706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:22.086544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:22.086981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:22.087964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:22.088223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:22.230920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:22.239611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-503c61d0-8836-4e7d-8714-b0376a09b0c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.166986 Jul 20 14:51:22.240184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 262/524] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:22 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:22.316935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:22.317622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:22.324185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:22.324678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:22.325601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:22.325947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:22.463039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:22.470938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9825cfdb-7d51-435a-9784-be97b00cc3fa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.159130 Jul 20 14:51:22.471521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 263/525] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:22 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:23.256003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:23.256669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:23.265171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:23.265606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:23.266420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:23.267117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:23.428184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:23.437489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d4c0e8fb-6e37-4ee8-b714-91000da1c1d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.185192 Jul 20 14:51:23.443087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 263/526] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:23.489415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:23.489957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:23.497207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:23.498046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:23.498888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:23.499599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:23.504346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=93 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:23.639315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:23.649516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4fc35e78-07d8-4d0b-82b8-d42adb071758 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.165635 Jul 20 14:51:23.650129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 264/527] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:23 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:23.651647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:51:23.652041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 14:51:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 14:51:24.456631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:24.457327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:24.463697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:24.464144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:24.464913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:24.465346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:24.614940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:24.622572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fdfd221d-6883-4ce4-8bda-5417ed3add84 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.170580 Jul 20 14:51:24.623147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 264/528] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:24.666871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:24.667741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:24.674097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:24.675972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:24.675972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:24.675972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:24.815066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:24.822879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-43d9a8fb-aee0-46fb-8751-fd318b03c422 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.159816 Jul 20 14:51:24.823419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 265/529] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:24 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:51:25.640246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:25.640829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:25.649910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:25.650358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:25.651162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:25.651597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:25.810061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:25.814871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-4bcf70e4-36fe-4027-bf14-19bbf37cd728 req-f2a13eea-e1b3-4f26-a3cc-10bebb2d60ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "192968a0-0654-43f6-8846-3e1bbb536fe5", "name": "network-vif-plugged", "status": "completed", "tag": "dd51169e-9f1a-4269-9378-37fbd6dec56a"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:25.819867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-89ff9f0b-d2ee-49c1-bd1d-236cc898f0db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1371 microversion: 2.1 time: 0.183460 Jul 20 14:51:25.821125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 265/530] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1371 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:25.823737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-4bcf70e4-36fe-4027-bf14-19bbf37cd728 req-f2a13eea-e1b3-4f26-a3cc-10bebb2d60ad service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:25.824330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-4bcf70e4-36fe-4027-bf14-19bbf37cd728 req-f2a13eea-e1b3-4f26-a3cc-10bebb2d60ad service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:25.838964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-4bcf70e4-36fe-4027-bf14-19bbf37cd728 req-f2a13eea-e1b3-4f26-a3cc-10bebb2d60ad service nova] Creating event network-vif-plugged:dd51169e-9f1a-4269-9378-37fbd6dec56a for instance 192968a0-0654-43f6-8846-3e1bbb536fe5 on hv-cinder-84902 Jul 20 14:51:25.840000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:25.841320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:25.844759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-4bcf70e4-36fe-4027-bf14-19bbf37cd728 req-f2a13eea-e1b3-4f26-a3cc-10bebb2d60ad service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033469 Jul 20 14:51:25.845378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 266/531] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:51:25.848219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:25.848721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:25.849525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:25.849958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:26.010534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:26.020571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-120a4780-49a8-490b-920e-fdf54fd6e8a8 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.184353 Jul 20 14:51:26.021138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 266/532] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:25 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:26.838968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:26.840244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:26.848742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:26.849484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:26.850511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:26.851083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:27.003186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:27.011106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7321b135-850a-4212-a3d2-8bec75b9d8c3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1564 microversion: 2.1 time: 0.176828 Jul 20 14:51:27.011720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 267/533] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1564 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:27.031329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:27.031969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:27.038769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:27.039007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:27.039278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:27.039856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:27.040023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:27.040672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:27.046925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:27.047413 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:27.048381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:27.048831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:27.170056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e033488b-276c-4acc-8d2f-759f7e2194d8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.143491 Jul 20 14:51:27.174853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 267/534] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 14:51:27 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 284 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:27.237050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:27.246068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b5108752-6495-4837-b19e-d2e517962b53 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.211741 Jul 20 14:51:27.247140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 268/535] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:27 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:28.229730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-168e8f30-33ce-4cb6-b29f-2fcbee546256 req-76290b3c-8c02-4683-a526-6e13c01005f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "75115162-3a7f-4a94-a703-190f8fdbc4c8", "tag": "94ca1675-607f-45ff-a048-c7df5f9c493a"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:28.238500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-168e8f30-33ce-4cb6-b29f-2fcbee546256 req-76290b3c-8c02-4683-a526-6e13c01005f4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:28.239013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-168e8f30-33ce-4cb6-b29f-2fcbee546256 req-76290b3c-8c02-4683-a526-6e13c01005f4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:28.250767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-168e8f30-33ce-4cb6-b29f-2fcbee546256 req-76290b3c-8c02-4683-a526-6e13c01005f4 service nova] Creating event network-changed:94ca1675-607f-45ff-a048-c7df5f9c493a for instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 on hv-cinder-84902 Jul 20 14:51:28.258963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-168e8f30-33ce-4cb6-b29f-2fcbee546256 req-76290b3c-8c02-4683-a526-6e13c01005f4 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033826 Jul 20 14:51:28.259704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 268/536] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:28.266463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:28.267232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:28.274442 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:28.274894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:28.275718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:28.276151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:28.428078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:28.435975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-19964212-a9e5-462e-b9ca-776ee9715b03 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1314 microversion: 2.1 time: 0.174779 Jul 20 14:51:28.436531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 269/537] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:28 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1314 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:29.455093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:29.460281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:29.472951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:29.473279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:29.474048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:29.474515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:29.504381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:29.639702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:29.648364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aa318768-a0ee-4e5f-b4bb-eb80579ef47d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1498 microversion: 2.1 time: 0.198140 Jul 20 14:51:29.649271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 269/538] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:29 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1498 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:51:30.239623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:30.240229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:30.243762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=70,nova_api:INSERT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:30.247671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:30.248156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:30.248930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:30.249370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:30.277355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:51:30.374660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ad3387d2-da83-492c-a563-8c6292ddcc84 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "DELETE /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 204 len: 0 microversion: 2.1 time: 0.138581 Jul 20 14:51:30.375318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 270/539] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:51:30 2022] DELETE /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:51:30.385536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:30.386224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:30.403500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:30.404083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:30.405107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:30.405653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:30.545594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:30.553649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2a970616-b24d-41f9-855d-d42b6f598527 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.173521 Jul 20 14:51:30.554392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 270/540] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:30 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:51:31.573802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:31.574499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:31.581841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:31.582279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:31.583073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:31.583508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:31.729093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:31.739081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-58ab80c6-d4e6-425c-bbf3-bf999cb5822e tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.169567 Jul 20 14:51:31.739682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 271/541] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:31 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:32.760423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:32.761248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:32.769859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:32.770299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:32.771076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:32.771505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:32.904148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:32.912221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bd3e1b59-357b-45eb-acdd-444e2efc6ddf tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.158308 Jul 20 14:51:32.912897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 271/542] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:32 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:33.929765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:33.930421 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:33.937621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:33.938191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:33.939000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:33.939472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:34.077360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:34.087005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e8ba75d1-ebeb-4823-ad9f-21d96a008b15 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.160791 Jul 20 14:51:34.087285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 272/543] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:33 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:35.112142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:35.112776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:35.122896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:35.123457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:35.124598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:35.125238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:35.277365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:35.285679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0a13b8f-c7b9-4150-9800-9564acf5e498 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.180076 Jul 20 14:51:35.286254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 272/544] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:35 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:36.303196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:36.303803 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:36.311957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:36.312466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:36.313427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:36.313939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:36.453032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:36.461569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-009a533b-af2d-4445-8195-dd1720462b80 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.162298 Jul 20 14:51:36.462153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 273/545] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:36 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:37.478495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:37.479126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:37.481085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=50 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:37.487792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:37.488232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:37.489002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:37.489452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:37.640356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:37.649234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8536cda6-4710-45d7-9302-6d300df88d66 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.174588 Jul 20 14:51:37.649804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 273/546] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:37 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:38.667140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:38.667713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:38.674905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:38.675385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:38.676157 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:38.676565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:38.811596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:38.819516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-26a42a38-57ff-4a34-951e-9416a61b39c0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.156282 Jul 20 14:51:38.820097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 274/547] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:38 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:39.838354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:39.838993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:39.846034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:39.846626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:39.847391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:39.847824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:39.980492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:39.989297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d6a4fc12-70f2-465f-896c-10ace41db491 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1504 microversion: 2.1 time: 0.154946 Jul 20 14:51:39.989901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 274/548] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:39 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:40.493683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-21c2a67e-8081-4c85-940b-c3ff8b870769 req-d3e09a92-e959-4c07-bf76-7d2c58e09fd4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "192968a0-0654-43f6-8846-3e1bbb536fe5", "name": "network-vif-unplugged", "status": "completed", "tag": "dd51169e-9f1a-4269-9378-37fbd6dec56a"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:40.504189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-21c2a67e-8081-4c85-940b-c3ff8b870769 req-d3e09a92-e959-4c07-bf76-7d2c58e09fd4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:40.504663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-21c2a67e-8081-4c85-940b-c3ff8b870769 req-d3e09a92-e959-4c07-bf76-7d2c58e09fd4 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:40.518714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-21c2a67e-8081-4c85-940b-c3ff8b870769 req-d3e09a92-e959-4c07-bf76-7d2c58e09fd4 service nova] Creating event network-vif-unplugged:dd51169e-9f1a-4269-9378-37fbd6dec56a for instance 192968a0-0654-43f6-8846-3e1bbb536fe5 on hv-cinder-84902 Jul 20 14:51:40.525301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-21c2a67e-8081-4c85-940b-c3ff8b870769 req-d3e09a92-e959-4c07-bf76-7d2c58e09fd4 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036476 Jul 20 14:51:40.525883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 275/549] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:40 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:40.854207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-d6c63ed3-1ad5-4bfc-9cff-ca2ce232ee36 req-85e0b89b-64f1-4f9b-9d29-b5eeae560727 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "192968a0-0654-43f6-8846-3e1bbb536fe5", "name": "network-vif-deleted", "tag": "dd51169e-9f1a-4269-9378-37fbd6dec56a"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:40.862828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d6c63ed3-1ad5-4bfc-9cff-ca2ce232ee36 req-85e0b89b-64f1-4f9b-9d29-b5eeae560727 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:40.863340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d6c63ed3-1ad5-4bfc-9cff-ca2ce232ee36 req-85e0b89b-64f1-4f9b-9d29-b5eeae560727 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:40.876505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-d6c63ed3-1ad5-4bfc-9cff-ca2ce232ee36 req-85e0b89b-64f1-4f9b-9d29-b5eeae560727 service nova] Creating event network-vif-deleted:dd51169e-9f1a-4269-9378-37fbd6dec56a for instance 192968a0-0654-43f6-8846-3e1bbb536fe5 on hv-cinder-84902 Jul 20 14:51:40.887269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-d6c63ed3-1ad5-4bfc-9cff-ca2ce232ee36 req-85e0b89b-64f1-4f9b-9d29-b5eeae560727 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031638 Jul 20 14:51:40.887269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 275/550] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:41.007506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:41.008697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:41.016192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:41.016993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:41.017981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:41.018601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:41.057082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:41.057082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:41.057308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lazy-loading 'fault' on Instance uuid 192968a0-0654-43f6-8846-3e1bbb536fe5 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:51:41.105549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:41.113254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dd43e0d2-951b-4f7b-91bd-41433a2247a9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1304 microversion: 2.1 time: 0.110806 Jul 20 14:51:41.113818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 276/551] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:41 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1304 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:42.130669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:42.131429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:42.138120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:42.138568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:42.139376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:42.139840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:42.171741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:42.172164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:42.173149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lazy-loading 'fault' on Instance uuid 192968a0-0654-43f6-8846-3e1bbb536fe5 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:51:42.215875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:42.223929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba81597d-737b-4fe2-b20e-b78f823f9209 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 200 len: 1304 microversion: 2.1 time: 0.097546 Jul 20 14:51:42.224456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 276/552] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:42 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 1304 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:43.241076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:43.241714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 192968a0-0654-43f6-8846-3e1bbb536fe5] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:43.248824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:43.249269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:43.250039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:43.250526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:43.268857 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] HTTP exception thrown: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Request method failure captured: Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 HTTP/1.1 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BZWA81Nu-paVyGLX5o5Uwc_QaqEq2IG1T1oOzhy60-J0qW5CrNqiuGwrrLqdbnC7VLRfb_cp7sLeluBrIn9D18iC9eHKKHvJ-4I5J1ZwL06-nsLVc7QilI6vC9NJPPg0QfJ1iuFai07EVb1hFKeaLrIprJpyZLFlqLfJ3TbT3Tv_xc Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 114da792d3f54f38b215aaeea69fd454 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesActionsTest-1109141095 Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:51:43.270526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_114da792d3f54f38b215aaeea69fd454"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/114da792d3f54f38b215aaeea69fd454"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/114da792d3f54f38b215aaeea69fd454"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/114da792d3f54f38b215aaeea69fd454"}]}] Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesActionsTest-1109141095 Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 114da792d3f54f38b215aaeea69fd454 Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesActionsTest-1109141095 Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesActionsTest-1109141095-project Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: c29795aae948425b87a6852b41a80c5d Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesActionsTest-1109141095-project Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. Jul 20 14:51:43.271946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '192968a0-0654-43f6-8846-3e1bbb536fe5'} Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:51:43.272769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. Jul 20 14:51:43.274210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:51:43.276344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Returning 404 to user: Instance 192968a0-0654-43f6-8846-3e1bbb536fe5 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:51:43.276344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3aef0d0d-86f6-4be2-ae81-202f12e68ac0 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5" status: 404 len: 111 microversion: 2.1 time: 0.034824 Jul 20 14:51:43.276344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 277/553] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:43 2022] GET /compute/v2.1/servers/192968a0-0654-43f6-8846-3e1bbb536fe5 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:51:43.376937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-394251194", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "networks": [{"uuid": "4f187311-3105-4dca-b98e-c91745d78e9f"}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:43.430997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] validate_networks() for [('4f187311-3105-4dca-b98e-c91745d78e9f', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:51:43.791677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:51:43.792126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:51:43.792597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:51:43.793004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:51:43.793409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:51:43.804534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:43.804975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:43.805736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:43.806976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:43.826256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Getting quotas for project 114da792d3f54f38b215aaeea69fd454. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:51:43.830756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Getting quotas for user c29795aae948425b87a6852b41a80c5d and project 114da792d3f54f38b215aaeea69fd454. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:51:43.842975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:51:43.845414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:51:43.892963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c5ce6ae9-6e7a-461c-a25b-128f6c3c3205 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.519926 Jul 20 14:51:43.893636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 277/554] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:51:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 521 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:51:43.904649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-88fb1d62-c3f4-49a6-b862-8f1bb9d1bd50 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:43.905351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-88fb1d62-c3f4-49a6-b862-8f1bb9d1bd50 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:43.963025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-88fb1d62-c3f4-49a6-b862-8f1bb9d1bd50 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:43.970259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-88fb1d62-c3f4-49a6-b862-8f1bb9d1bd50 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1213 microversion: 2.1 time: 0.070022 Jul 20 14:51:43.971276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 278/555] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:43 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1213 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:44.990175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:44.990808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:44.998335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:44.998779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:44.999582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:45.000018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:45.070621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:45.078913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f3f1a26-e05a-4c9a-9314-c9f44280415b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1217 microversion: 2.1 time: 0.094418 Jul 20 14:51:45.079525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 278/556] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:44 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1217 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:46.101749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:46.102560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:46.111706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:46.112295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:46.113231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:46.113677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:46.193636 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:46.202123 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b2cf91f9-f5b3-4dd8-8e4b-c4f5f86b2922 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1265 microversion: 2.1 time: 0.104531 Jul 20 14:51:46.202672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 279/557] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:46 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1265 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:47.224141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:47.225548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:47.238289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:47.240111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:47.241356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:47.243006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:47.428133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:47.436776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4ca5c531-3618-438d-af92-7426af877e59 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.217905 Jul 20 14:51:47.437785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 279/558] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:47 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:47.967351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-a7fde579-d8f3-4e15-96d3-b630a8cd3d6d req-ad0d84db-a133-4454-a888-3d11f40a2b86 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7e5d2a8c-43a3-42ce-a965-9fc7310e1b83", "tag": "67f3fb84-3c96-442c-9fd3-f6ee7d1345a3"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:47.981176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-a7fde579-d8f3-4e15-96d3-b630a8cd3d6d req-ad0d84db-a133-4454-a888-3d11f40a2b86 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:47.982024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-a7fde579-d8f3-4e15-96d3-b630a8cd3d6d req-ad0d84db-a133-4454-a888-3d11f40a2b86 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:47.996748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-a7fde579-d8f3-4e15-96d3-b630a8cd3d6d req-ad0d84db-a133-4454-a888-3d11f40a2b86 service nova] Creating event network-changed:67f3fb84-3c96-442c-9fd3-f6ee7d1345a3 for instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 on hv-cinder-84902 Jul 20 14:51:48.006136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-a7fde579-d8f3-4e15-96d3-b630a8cd3d6d req-ad0d84db-a133-4454-a888-3d11f40a2b86 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042402 Jul 20 14:51:48.007147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 280/559] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:48.455172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:48.456611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:48.463084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:48.463684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:48.464474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:48.464906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:48.605099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:48.613474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d4b5a5f4-9e0b-400c-b5b8-ce1ea5e69775 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.162612 Jul 20 14:51:48.614041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 280/560] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:48 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:49.636266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:49.637364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:49.648033 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:49.648777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:49.650024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:49.650727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:49.796256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:49.804671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-42f159dd-4269-4283-93fb-97764747beb9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.176287 Jul 20 14:51:49.805247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 281/561] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:49 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:50.825420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:50.826014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:50.832345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:50.832781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:50.833562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:50.833978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:50.967402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:50.975439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3abc849e-2406-4c09-b7da-ae8397c3b38d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.153852 Jul 20 14:51:50.976000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 281/562] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:50 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:51.993834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:51.994445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:52.002995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:52.003470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:52.004525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:52.004956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:52.029080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:52.162538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:52.170209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5df89faa-7c60-4a6a-8a2d-ff195833384b tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.181620 Jul 20 14:51:52.170798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 282/563] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:51 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:53.187113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:53.187926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:53.194703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:53.195167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:53.196114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:53.196704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:53.351592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:53.360422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ee268a67-3d9a-4aa0-bd93-6df76581bbba tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.177111 Jul 20 14:51:53.361553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 282/564] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:53 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:54.377493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:54.378101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:54.380510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:54.385541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:54.385978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:54.386768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:54.387224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:54.534234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:54.541816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4a2e062f-c0a6-4574-90c8-d69884827baa tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.168265 Jul 20 14:51:54.542391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 283/565] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:54 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:55.020373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-2e9437d4-22d6-4a06-8e82-e511aab8422f req-fbf37ca7-2c0d-4aef-9de1-2cf4946071af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e5d2a8c-43a3-42ce-a965-9fc7310e1b83", "name": "network-vif-plugged", "status": "completed", "tag": "67f3fb84-3c96-442c-9fd3-f6ee7d1345a3"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:51:55.025069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:55.025740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:51:55.033193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2e9437d4-22d6-4a06-8e82-e511aab8422f req-fbf37ca7-2c0d-4aef-9de1-2cf4946071af service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:55.033890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2e9437d4-22d6-4a06-8e82-e511aab8422f req-fbf37ca7-2c0d-4aef-9de1-2cf4946071af service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:55.052274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-2e9437d4-22d6-4a06-8e82-e511aab8422f req-fbf37ca7-2c0d-4aef-9de1-2cf4946071af service nova] Creating event network-vif-plugged:67f3fb84-3c96-442c-9fd3-f6ee7d1345a3 for instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 on hv-cinder-84902 Jul 20 14:51:55.057673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-2e9437d4-22d6-4a06-8e82-e511aab8422f req-fbf37ca7-2c0d-4aef-9de1-2cf4946071af service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040941 Jul 20 14:51:55.058632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 283/566] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:51:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:51:55.561912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:55.562628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:55.573043 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:55.573491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:55.574270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:55.574705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:55.729519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:55.748020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a9c97748-df56-4f82-873d-1d667cdf8646 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.189778 Jul 20 14:51:55.749183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 284/567] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:55 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:56.766755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:56.767793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:56.775476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:56.776101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:56.778091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:56.778687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:57.132498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:57.140988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6129c2f3-f8b5-42e0-bda3-402d90300cf3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1313 microversion: 2.1 time: 0.378922 Jul 20 14:51:57.152285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 284/568] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:56 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1313 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:58.161843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:58.163789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:58.180144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:58.180598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:58.181611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:58.182036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:58.464039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:58.477301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0d4f96f4-bfcd-438c-bbc1-37f1042584c4 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1496 microversion: 2.1 time: 0.322011 Jul 20 14:51:58.501937 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 285/569] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:58 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1496 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:51:59.136798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:59.137300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:59.144352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:59.144798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:59.145660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:59.146112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:59.176116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:51:59.270808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab4e8f54-bd63-4e1c-905e-b58249febcee tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "DELETE /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 204 len: 0 microversion: 2.1 time: 0.138000 Jul 20 14:51:59.272538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 285/570] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:51:59 2022] DELETE /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:51:59.282055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:51:59.282667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:51:59.290428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:59.290876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:59.291672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:51:59.292102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:51:59.551510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:51:59.565110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1bd6d00-2e93-436f-8e40-6aa5113908e3 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1502 microversion: 2.1 time: 0.286896 Jul 20 14:51:59.566184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 286/571] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:51:59 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1502 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:00.583075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:00.584025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:00.590346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:00.590806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:00.591629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:00.592241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:00.609067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:00.852235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:00.861557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfa28a1f-09b4-40b7-acad-ebe728d8c1d9 tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1502 microversion: 2.1 time: 0.282618 Jul 20 14:52:00.863202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 286/572] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:00 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1502 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:01.878135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:01.878758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:01.881249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:01.888256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:01.888805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:01.889613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:01.890059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:02.147013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c29795aae948425b87a6852b41a80c5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '114da792d3f54f38b215aaeea69fd454', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:02.160886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0b6f83b9-310b-4e05-9a75-6073cbbfe1bb tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 200 len: 1502 microversion: 2.1 time: 0.286386 Jul 20 14:52:02.161721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 287/573] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:01 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 1502 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:02.244973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-64258b9d-60e2-4285-a5b2-2eeded7fd1c9 req-2511d0c9-df8c-4dce-8789-98be7984304f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e5d2a8c-43a3-42ce-a965-9fc7310e1b83", "name": "network-vif-deleted", "tag": "67f3fb84-3c96-442c-9fd3-f6ee7d1345a3"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:02.264204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-64258b9d-60e2-4285-a5b2-2eeded7fd1c9 req-2511d0c9-df8c-4dce-8789-98be7984304f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:02.264659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-64258b9d-60e2-4285-a5b2-2eeded7fd1c9 req-2511d0c9-df8c-4dce-8789-98be7984304f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:02.286346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-64258b9d-60e2-4285-a5b2-2eeded7fd1c9 req-2511d0c9-df8c-4dce-8789-98be7984304f service nova] Creating event network-vif-deleted:67f3fb84-3c96-442c-9fd3-f6ee7d1345a3 for instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 on hv-cinder-84902 Jul 20 14:52:02.307390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-64258b9d-60e2-4285-a5b2-2eeded7fd1c9 req-2511d0c9-df8c-4dce-8789-98be7984304f service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065720 Jul 20 14:52:02.308703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 287/574] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:03.178214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:03.178848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] [instance: 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:03.187591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:03.188048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:03.189021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:03.189664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:03.210265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] HTTP exception thrown: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Request method failure captured: Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 HTTP/1.1 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BZWA81Nu-paVyGLX5o5Uwc_QaqEq2IG1T1oOzhy60-J0qW5CrNqiuGwrrLqdbnC7VLRfb_cp7sLeluBrIn9D18iC9eHKKHvJ-4I5J1ZwL06-nsLVc7QilI6vC9NJPPg0QfJ1iuFai07EVb1hFKeaLrIprJpyZLFlqLfJ3TbT3Tv_xc Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 114da792d3f54f38b215aaeea69fd454 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-VolumesActionsTest-1109141095 Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:52:03.213704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_114da792d3f54f38b215aaeea69fd454"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/114da792d3f54f38b215aaeea69fd454"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/114da792d3f54f38b215aaeea69fd454"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/114da792d3f54f38b215aaeea69fd454"}]}] Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-VolumesActionsTest-1109141095 Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 114da792d3f54f38b215aaeea69fd454 Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-VolumesActionsTest-1109141095 Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-VolumesActionsTest-1109141095-project Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: c29795aae948425b87a6852b41a80c5d Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-VolumesActionsTest-1109141095-project Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. Jul 20 14:52:03.215353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '7e5d2a8c-43a3-42ce-a965-9fc7310e1b83'} Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:52:03.216173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. Jul 20 14:52:03.217456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:52:03.220347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] Returning 404 to user: Instance 7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:52:03.222056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-173910cb-4245-44e3-9492-be39827ff75d tempest-VolumesActionsTest-1109141095 tempest-VolumesActionsTest-1109141095-project] 10.222.0.15 "GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83" status: 404 len: 111 microversion: 2.1 time: 0.047336 Jul 20 14:52:03.223465 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 288/575] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:03 2022] GET /compute/v2.1/servers/7e5d2a8c-43a3-42ce-a965-9fc7310e1b83 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:52:04.762103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3daba628-c894-44b0-a1f8-d9b8e638618b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Action: 'create', calling method: .version_select at 0x7f7b15dfe4c0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1661908460", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH3wg3amgDXY4UGmXMM2RDbvqod1at/wHvV1khEf9YKHcWPvhSjK5g0OPqQcs7EqN9gPYkMNg7fHcG8T+sQgIOlhtt08tPci3rxt2WOaCgoDNyjnkHDhTFmwqQvAzZ+nWA=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:04.783803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-3daba628-c894-44b0-a1f8-d9b8e638618b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Getting quotas for project 779e3cc39cf14719ba34b3d12cac5a01. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:52:04.789416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-3daba628-c894-44b0-a1f8-d9b8e638618b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Getting quotas for user 58016f5cae3d45eab6f9caf161e65c4c and project 779e3cc39cf14719ba34b3d12cac5a01. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:52:04.820686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3daba628-c894-44b0-a1f8-d9b8e638618b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.266271 Jul 20 14:52:04.821383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 288/576] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:52:04 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:08.705285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:08.705939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:08.713902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:08.714350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:08.715237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:08.715791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:08.746148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e0ee39e2-e2c8-4b65-ae4c-c00c504aeead tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/ips" status: 200 len: 152 microversion: 2.20 time: 0.044738 Jul 20 14:52:08.746741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 289/577] 10.222.0.15 () {60 vars in 1288 bytes} [Wed Jul 20 14:52:08 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/ips => generated 152 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:52:12.142604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:12.143430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:12.145834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:12.153965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:12.154726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:12.156017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:12.156467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:12.215895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'flavor' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:52:14.800909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:22.210034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:22.304683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=22,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:22.625494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=5 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:22.628319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=39 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:22.647881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:22.648145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:22.656039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:22.656488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:22.657276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:22.657855 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:22.685756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2587da5e-782a-43c1-846b-967a84b97462 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/ips" status: 200 len: 149 microversion: 2.1 time: 0.072980 Jul 20 14:52:22.687024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 290/578] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:52:22 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/ips => generated 149 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:22.713724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:22.714223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:22.721050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:22.721507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:22.722463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:22.723241 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:22.761163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ca2c1dc7-4607-444c-a3e1-610cc3e528a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/ips" status: 200 len: 147 microversion: 2.1 time: 0.054078 Jul 20 14:52:22.761979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 291/579] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:52:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/ips => generated 147 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:24.144640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a29f28ea-3635-4cf2-bc86-5a98db9dbb5e"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:24.145710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:24.156290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:24.156825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:24.157943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:24.158370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:24.485682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a82a0f6c-a71a-44fd-aca1-1ea297c210eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.344457 Jul 20 14:52:24.486571 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 292/580] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:52:24 2022] POST /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 194 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:52:26.162192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "647e4365-be89-4100-91f7-a1ee50e4487a"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:26.164622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:26.173783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:26.174472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:26.175231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:26.175809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:26.495682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2857160e-e4c5-4711-a8b9-402dcbe624df tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.338941 Jul 20 14:52:26.496663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 293/581] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:52:26 2022] POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 194 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:52:27.822315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9438a29-b443-4bc1-b010-23783b3872a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action" status: 202 len: 0 microversion: 2.20 time: 15.684506 Jul 20 14:52:27.823149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 289/582] 10.222.0.15 () {62 vars in 1318 bytes} [Wed Jul 20 14:52:12 2022] POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action => generated 0 bytes in 15686 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:52:27.838116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:27.839511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:27.850328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:27.850891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:27.851807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:27.852304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:28.043684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:28.052415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:28.053894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:28.055089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f1ca6d59-41d7-4726-bf45-9f53fa817fb7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.226828 Jul 20 14:52:28.063499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 294/583] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:29.083930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:29.085594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:29.094399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:29.095028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:29.096008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:29.096518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:29.263020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:29.274707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:29.277266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:29.281085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a711e589-b598-4757-a43e-372683f5fbb9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.200758 Jul 20 14:52:29.289578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 290/584] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:30.300961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:30.301586 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:30.310069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:30.310069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:30.310069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:30.310439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:30.469933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:30.494799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:30.497193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:30.498653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a2932702-ec60-4f36-a9b6-035df1b1be13 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.201759 Jul 20 14:52:30.500045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 295/585] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:31.528813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:31.529601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:31.539197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:31.539962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:31.541125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:31.541740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:31.708585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:31.719107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:31.720678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:31.721950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc9df49c-d94e-4467-b03b-54de05139e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.198027 Jul 20 14:52:31.722776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 291/586] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:32.745128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:32.745896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:32.753453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:32.753894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:32.754705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:32.755180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:32.898246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:32.909608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:32.911159 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:32.912298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1f53fa6a-d3a6-4085-9f3e-30bcc8171fdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.171992 Jul 20 14:52:32.912824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 296/587] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:33.256576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eb5026e8-906c-40a4-806e-1c4d123239a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: .version_select at 0x7f7b15e4b160>, body: {"keypair": {"name": "tempest-keypair-1341317015", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG9T+Hvb0edc1pKR26hzVYssrvgH7qog0XN88KFvyxauoHibJNqWoKW131wigX0v8EfzGXOo4s7zlg8wYpGuz+vIdpqNFTfVXdbWmseM43sRTBq2wKEErJCW5MWQJWBnOg=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:33.272127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-eb5026e8-906c-40a4-806e-1c4d123239a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:52:33.276529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-eb5026e8-906c-40a4-806e-1c4d123239a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:52:33.303396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eb5026e8-906c-40a4-806e-1c4d123239a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.050761 Jul 20 14:52:33.303912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 292/588] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 14:52:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:33.930060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:33.930545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:33.937766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:33.938209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:33.939066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:33.939521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:34.145643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:34.156710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:34.163526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:34.166144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-97860e5f-680b-46a4-9f49-e9c07b4aed97 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.239211 Jul 20 14:52:34.177834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 297/589] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:34.724678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1363871014", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "bb36c589-f60d-4c91-8345-5777287e4f0d"}], "security_groups": [{"name": "tempest-securitygroup--49647537"}], "key_name": "tempest-keypair-1341317015", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:34.843412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] validate_networks() for [('bb36c589-f60d-4c91-8345-5777287e4f0d', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:52:35.193369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:35.194316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:35.204595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:35.205240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:35.206380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:35.207006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:35.237234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:52:35.237656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:52:35.238157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:52:35.238559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:52:35.238965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:52:35.248300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:35.248830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:35.249624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:35.250025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:35.267077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:52:35.271037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:52:35.282575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:52:35.284917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:52:35.338605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-533da658-91a4-4008-82d2-f0b36b0f3612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.622197 Jul 20 14:52:35.339290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 293/590] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:52:34 2022] POST /compute/v2.1/servers => generated 400 bytes in 624 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 20 14:52:35.354374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca75a967-0233-437a-bcb9-466d87bb8546 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:35.359305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ca75a967-0233-437a-bcb9-466d87bb8546 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:35.371202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:35.380205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:35.381686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:35.382831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2486172b-8f34-4db2-8e96-45fed707c453 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.200926 Jul 20 14:52:35.383930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 298/591] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:35.419971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ca75a967-0233-437a-bcb9-466d87bb8546 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:35.428174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ca75a967-0233-437a-bcb9-466d87bb8546 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1242 microversion: 2.1 time: 0.081397 Jul 20 14:52:35.428847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 294/592] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:35 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1242 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:36.404549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:36.405229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:36.411897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:36.412424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:36.413274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:36.413914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:36.446780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:36.447421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:36.461019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:36.461722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:36.462793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:36.463387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:36.548585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:36.557182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb947319-f8a5-413b-8e0b-0279c60528d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1302 microversion: 2.1 time: 0.114383 Jul 20 14:52:36.567861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 295/593] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:36 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1302 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:36.642327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:36.666202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:36.669255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:36.672201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-df9d05a0-0170-4476-9bb5-f03f7db22423 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.273503 Jul 20 14:52:36.673606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 299/594] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:37.575151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:37.576377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:37.582616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:37.583149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:37.583944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:37.584385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:37.707174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:37.708455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:37.715795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:37.716414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:37.717443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:37.718210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:37.793761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:37.801720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cc6aaa4f-cd4f-4a51-aa39-3c4a9549ca74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.230433 Jul 20 14:52:37.802301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 296/595] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:37 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:37.930132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:37.940487 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:37.942134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:37.943458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b09167b3-e77e-4177-bbc7-7980682a53d2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.242675 Jul 20 14:52:37.944111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 300/596] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:38.238228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-a5a230b2-d577-4125-a846-08b39e16e6d1 req-0e7b016f-1f88-4ac1-ae6b-b2e19c70278e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88eb690a-bcac-4a8e-82df-86f0d0280558", "tag": "288d0f95-2b49-4578-b851-0f58b382c611"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:38.255639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-a5a230b2-d577-4125-a846-08b39e16e6d1 req-0e7b016f-1f88-4ac1-ae6b-b2e19c70278e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:38.256084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-a5a230b2-d577-4125-a846-08b39e16e6d1 req-0e7b016f-1f88-4ac1-ae6b-b2e19c70278e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:38.291494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-a5a230b2-d577-4125-a846-08b39e16e6d1 req-0e7b016f-1f88-4ac1-ae6b-b2e19c70278e service nova] Creating event network-changed:288d0f95-2b49-4578-b851-0f58b382c611 for instance 88eb690a-bcac-4a8e-82df-86f0d0280558 on hv-cinder-84902 Jul 20 14:52:38.313222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-a5a230b2-d577-4125-a846-08b39e16e6d1 req-0e7b016f-1f88-4ac1-ae6b-b2e19c70278e service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079198 Jul 20 14:52:38.314479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 297/597] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:38.819038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:38.819964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:38.827317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:38.827783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:38.828588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:38.829026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:38.968089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:38.969517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:38.976876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:38.977578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:38.978968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:38.980084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:39.015694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:39.023648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1b418225-817e-4b3d-bb16-9a8fccfe2eb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.208364 Jul 20 14:52:39.024269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 301/598] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:38 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:39.133565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:39.142308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:39.143848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:39.145076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-218cd9df-f751-4724-88d3-b7fa205676cd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.186856 Jul 20 14:52:39.145744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 298/599] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:40.040747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:40.041409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:40.048563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.048937 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.049675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.050129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.165055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:40.165658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:40.172201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.172653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.173437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.173879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.194393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:40.202540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-71cbc557-f3af-4109-8801-810511da4b2a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.165517 Jul 20 14:52:40.203308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 302/600] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:40 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:40.313643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:40.323357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:40.326694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:40.328600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-835c5366-7987-4915-b934-7de61f99f3a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.167675 Jul 20 14:52:40.329173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 299/601] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:40.349753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2020372696", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "networks": [{"uuid": "3e43663e-3658-4f8c-a038-ce7c5d38739c"}]}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:40.406418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.network.neutron [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] validate_networks() for [('3e43663e-3658-4f8c-a038-ce7c5d38739c', None, None, None, None, None)] {{(pid=124397) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:52:40.535086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2928b281-5451-42dd-97e8-755c9ae07e72 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:40.544762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2928b281-5451-42dd-97e8-755c9ae07e72 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.555440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2928b281-5451-42dd-97e8-755c9ae07e72 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.625538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2928b281-5451-42dd-97e8-755c9ae07e72 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Going to try to stop instance {{(pid=124396) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 20 14:52:40.681959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2928b281-5451-42dd-97e8-755c9ae07e72 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action" status: 202 len: 0 microversion: 2.1 time: 0.150280 Jul 20 14:52:40.683175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 300/602] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:52:40 2022] POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 14:52:40.691988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:40.692990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:40.700195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.700896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.701981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:40.702602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:40.915724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:40.924758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dcd34a7a-9337-4214-8d4e-d1cd1da385f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.237283 Jul 20 14:52:40.925348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 301/603] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:40 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:41.137518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Flavor limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:52:41.138025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Image limits 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:52:41.138387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Flavor pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:52:41.138798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Image pref 0:0:0 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:52:41.139234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.virt.hardware [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124397) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:52:41.149291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.149737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.150854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.151324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.168838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Getting quotas for project 1cdab962e8b0462b8d1d28dcde703c15. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:52:41.173108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.quota [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Getting quotas for user bc2f1a0d431341d6a5d1a8c5de131ece and project 1cdab962e8b0462b8d1d28dcde703c15. Resources: {'cores', 'ram', 'instances'} {{(pid=124397) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:52:41.184985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Going to run 1 instances... {{(pid=124397) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:52:41.187353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124397) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:52:41.221414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:41.222023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:41.228352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.228777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.229539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.230924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.232403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e52b4574-37d6-4aa9-bc88-b395f2965a39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.886497 Jul 20 14:52:41.233416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 303/604] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:52:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 888 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:52:41.243520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-29ecc254-8090-4a5d-9839-a3da95985680 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:41.244244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-29ecc254-8090-4a5d-9839-a3da95985680 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:41.331304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-29ecc254-8090-4a5d-9839-a3da95985680 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:41.338612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-29ecc254-8090-4a5d-9839-a3da95985680 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1219 microversion: 2.1 time: 0.098819 Jul 20 14:52:41.348540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 304/605] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:41 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1219 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:41.352976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:41.353580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:41.360845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.361287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.362037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.362636 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.434857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:41.443767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0adaf920-4eb9-4a2b-b559-db4b6927e098 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.226618 Jul 20 14:52:41.444386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 302/606] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:41 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:41.515669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:41.527615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:41.529118 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:41.530300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1d86985c-67af-4449-8f72-e04782eb0be1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.181173 Jul 20 14:52:41.530867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 305/607] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:41.619818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1180353322", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1661908460", "security_groups": [{"name": "tempest-secgroup-smoke-1594139827"}], "networks": [{"uuid": "8a7c6fbf-a0ed-4aca-a67d-19466fbdc9ad"}], "block_device_mapping_v2": [{"uuid": "321df6e9-b538-406a-818c-e14afdfbaa46", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:41.747497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] validate_networks() for [('8a7c6fbf-a0ed-4aca-a67d-19466fbdc9ad', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 14:52:41.941923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:41.942648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:41.949102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.949640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.950653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:41.951098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:41.955549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:42.183085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.191973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0cead290-688a-4ce6-8faa-0b415db50d5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.253958 Jul 20 14:52:42.192592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 306/608] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:42.205125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 14:52:42.205535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 14:52:42.206027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 14:52:42.206423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 14:52:42.206821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 14:52:42.217515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.217950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.218882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.219337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.237860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Getting quotas for project 779e3cc39cf14719ba34b3d12cac5a01. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 14:52:42.242345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Getting quotas for user 58016f5cae3d45eab6f9caf161e65c4c and project 779e3cc39cf14719ba34b3d12cac5a01. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 14:52:42.301499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 14:52:42.303609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='321df6e9-b538-406a-818c-e14afdfbaa46',volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 14:52:42.358671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:42.359840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:42.369114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.370013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.371273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.371973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.449243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.458106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-28f88a4f-8afd-4a1e-8c6e-022f5059f3fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1223 microversion: 2.1 time: 0.104973 Jul 20 14:52:42.458878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 307/609] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1223 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:42.459408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72e083c2-3820-48f9-931d-4625b28d27c4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.843346 Jul 20 14:52:42.463453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:42.464066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:42.465682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 303/610] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 14:52:41 2022] POST /compute/v2.1/servers => generated 406 bytes in 850 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 14:52:42.470556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-227a19aa-94c7-4af7-91c9-73121ab43a22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:42.470733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.471116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.471262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-227a19aa-94c7-4af7-91c9-73121ab43a22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:42.471905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.472345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.570972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-227a19aa-94c7-4af7-91c9-73121ab43a22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.578435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-227a19aa-94c7-4af7-91c9-73121ab43a22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1089 microversion: 2.1 time: 0.111777 Jul 20 14:52:42.579155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 304/611] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:42 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1089 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:42.584661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:42.585745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:42.595246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.595700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.596488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:42.596921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:42.652295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.660706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2b75b0c7-9a8c-45fb-b672-66016e61a4b2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.200888 Jul 20 14:52:42.661273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 308/612] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:42 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:42.749402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.758851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.760634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:42.761906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5338a4b0-227a-4e80-bfc6-23ebdfe2b006 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.181437 Jul 20 14:52:42.762637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 305/613] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:43.210844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:43.211709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:43.220993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.221452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.222449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.223095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.373275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.381529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c49ddfdc-2ff6-422b-a1ae-454f20d563fb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.175756 Jul 20 14:52:43.382105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 309/614] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:43.477536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:43.478294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:43.485057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.485761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.486643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.487438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.560786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.568706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0b28de2c-a4e1-483a-885d-149d815eba99 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1271 microversion: 2.1 time: 0.095045 Jul 20 14:52:43.569279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 306/615] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:43 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1271 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:43.596049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:43.596709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:43.603189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.603638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.604412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.604830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.678235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:43.679092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:43.684815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.688511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.689239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.690136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.690443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.695591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-700231cc-06b2-4439-b204-4f1c4345f618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1187 microversion: 2.1 time: 0.103368 Jul 20 14:52:43.696206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 310/616] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:43 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1187 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:43.782168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:43.782814 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:43.790058 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.790489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.791342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:43.791776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:43.859232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.867566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-994d7a31-f624-429c-bf99-0fe4af55ec45 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.193529 Jul 20 14:52:43.868131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 307/617] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:43 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:43.949806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.958806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.960334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:43.961490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-515bfd22-9b7a-48b8-9ca8-2cc1b6a7bbf2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.183576 Jul 20 14:52:43.962049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 311/618] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:44.398913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:44.399553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:44.406330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.406767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.407581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.408013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.551998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:44.560578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4e5364c6-21b0-496e-b86f-39c9bf0025a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.165438 Jul 20 14:52:44.568905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 308/619] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:44.586618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:44.587509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:44.596439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.596998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.597785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.598221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.672458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:44.683874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8a515ae0-50c1-4412-bfb6-3daad10e95d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1279 microversion: 2.1 time: 0.102064 Jul 20 14:52:44.685267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 312/620] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:44 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1279 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:44.713053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:44.713662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:44.728807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.730424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.733379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.734037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.834166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:44.842023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e59c3ae9-1993-424a-8ebf-93d5318c19f2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1195 microversion: 2.1 time: 0.133075 Jul 20 14:52:44.842788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 309/621] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:44 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1195 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:44.885731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:44.886727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:44.895466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.896302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.897757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.898592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.980431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:44.981283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:44.987794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.988348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:44.989177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:44.989707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.011102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:45.145031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:45.155586 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b1cb05f4-762f-4918-98ad-93037c416454 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.273985 Jul 20 14:52:45.164810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 313/622] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:44 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:45.264522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:45.273285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:45.275283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:45.276688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dd578804-eb69-498f-8e84-73e9214879ea tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.300221 Jul 20 14:52:45.277696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 310/623] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:45.577935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:45.578555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:45.585101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.585849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.586800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.587948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.706140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:45.707875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:45.717665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.718908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.720016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.721189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.838039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:45.850651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-766a74b7-a3e2-4d16-b36a-74d21b43e53a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.275535 Jul 20 14:52:45.865963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 314/624] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:45.873674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:45.874813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:45.888398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.889531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:45.890883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:45.891610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.003181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.021798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d53b3344-ed42-4910-b574-1e9b0914168e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.322503 Jul 20 14:52:46.022366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 311/625] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:45 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:46.028502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-fd2d073c-b375-44d2-8bb6-c84b30aa24bc req-fec68032-62ff-440b-ad6a-e1f702de6c48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88eb690a-bcac-4a8e-82df-86f0d0280558", "name": "network-vif-plugged", "status": "completed", "tag": "288d0f95-2b49-4578-b851-0f58b382c611"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:46.039686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-fd2d073c-b375-44d2-8bb6-c84b30aa24bc req-fec68032-62ff-440b-ad6a-e1f702de6c48 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.041018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-fd2d073c-b375-44d2-8bb6-c84b30aa24bc req-fec68032-62ff-440b-ad6a-e1f702de6c48 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.069958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.083982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-18f56e56-6f10-4fea-b184-4d2305611aff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1205 microversion: 2.1 time: 0.216502 Jul 20 14:52:46.094517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 315/626] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:45 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1205 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:46.118281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-fd2d073c-b375-44d2-8bb6-c84b30aa24bc req-fec68032-62ff-440b-ad6a-e1f702de6c48 service nova] Creating event network-vif-plugged:288d0f95-2b49-4578-b851-0f58b382c611 for instance 88eb690a-bcac-4a8e-82df-86f0d0280558 on hv-cinder-84902 Jul 20 14:52:46.127494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-fd2d073c-b375-44d2-8bb6-c84b30aa24bc req-fec68032-62ff-440b-ad6a-e1f702de6c48 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103680 Jul 20 14:52:46.134269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 312/627] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:46.173620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:46.174246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:46.181877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.183818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.184679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.185127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.296616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:46.297327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:46.317078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.317461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.318317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.318774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.565114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.572972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-958e266a-7d05-4e3e-8d6e-348d78ab399f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.404439 Jul 20 14:52:46.583475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 316/628] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:46 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:46.624535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.640370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.642168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:46.643412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3134082-1d30-4cfc-8efb-0ed89bb8e923 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.351406 Jul 20 14:52:46.644162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 313/629] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:46.856809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-7140ba76-bbf8-43e5-8a42-6c9c1f842742 req-e176cfdc-ded0-46bf-8949-589d26c6b384 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e827c810-220b-4fe1-9d16-01bc8372b933", "tag": "1f670f13-f9d2-4b6a-b0ce-7144e9adc59f"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:46.866634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-7140ba76-bbf8-43e5-8a42-6c9c1f842742 req-e176cfdc-ded0-46bf-8949-589d26c6b384 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.867204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-7140ba76-bbf8-43e5-8a42-6c9c1f842742 req-e176cfdc-ded0-46bf-8949-589d26c6b384 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.870112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:46.870726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:46.878003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.878705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.879925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:46.880394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:46.884172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-7140ba76-bbf8-43e5-8a42-6c9c1f842742 req-e176cfdc-ded0-46bf-8949-589d26c6b384 service nova] Creating event network-changed:1f670f13-f9d2-4b6a-b0ce-7144e9adc59f for instance e827c810-220b-4fe1-9d16-01bc8372b933 on hv-cinder-84902 Jul 20 14:52:46.889337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-7140ba76-bbf8-43e5-8a42-6c9c1f842742 req-e176cfdc-ded0-46bf-8949-589d26c6b384 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036134 Jul 20 14:52:46.894642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 317/630] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:47.021764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.030585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b829b40-14d5-4e52-a701-30c174fa113a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.164126 Jul 20 14:52:47.031179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 314/631] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:47.042055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:47.042859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:47.052909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.053359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.054386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.054921 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.103841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:47.105287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:47.112588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.113065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.114316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.114784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.270941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.281356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-40a8a7dc-0900-4f3c-8876-ba2c8d8a72ef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.243720 Jul 20 14:52:47.282239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 318/632] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:47 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:47.322862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.333261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f937c995-8547-43fd-8111-7b83bd8794f3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.234468 Jul 20 14:52:47.333861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 315/633] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:47 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:47.388865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-4cf34982-2dd4-47f5-b512-75c000c8ad94 req-18b26300-d226-45a5-94fa-6d390dc18af7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:47.396648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-4cf34982-2dd4-47f5-b512-75c000c8ad94 req-18b26300-d226-45a5-94fa-6d390dc18af7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.397036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-4cf34982-2dd4-47f5-b512-75c000c8ad94 req-18b26300-d226-45a5-94fa-6d390dc18af7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.419510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-4cf34982-2dd4-47f5-b512-75c000c8ad94 req-18b26300-d226-45a5-94fa-6d390dc18af7 service nova] Creating event network-changed:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 14:52:47.425761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-4cf34982-2dd4-47f5-b512-75c000c8ad94 req-18b26300-d226-45a5-94fa-6d390dc18af7 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042567 Jul 20 14:52:47.426999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 319/634] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:47.590429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:47.591033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:47.594431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:47.598065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.598531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.599395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.599874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.662462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:47.663378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:47.670003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.670570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.671478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:47.672463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:47.782302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.790405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e6a1e3ae-f909-4431-943e-169e9b550540 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.203763 Jul 20 14:52:47.790977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 316/635] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:47 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:47.833603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.842189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.843779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:47.844974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2f9c1115-2658-46ef-b0fa-ac95163752c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.186780 Jul 20 14:52:47.845521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 320/636] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:48.048195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:48.048829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:48.055535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.055976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.058556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.058556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.298589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:48.300845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:48.313531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.315602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.317852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.320365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.340937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:48.349591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-22b5f25d-2526-43e8-bc04-db7096cfd041 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.305229 Jul 20 14:52:48.350465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 317/637] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:48.355232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:48.355958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:48.362650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.363197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.364066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.364599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.587428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:48.590509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:48.596707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-46faf9f7-bb6a-4cfa-92af-a1a273d819d5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.244918 Jul 20 14:52:48.597513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 318/638] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:48 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:48.598330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8dea7d75-1fd9-46ce-b48f-79e7cdee8019 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.304373 Jul 20 14:52:48.598913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 321/639] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:48 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:48.807723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:48.808331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:48.823232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.824038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.824817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.825243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.863269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:48.863896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:48.882192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.882902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:48.884089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:48.884691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.046486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.054706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30a4371a-07cc-4511-b9ef-40d38194cb20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1366 microversion: 2.1 time: 0.250793 Jul 20 14:52:49.055423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 319/640] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:48 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1366 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:49.079563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.088061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.089535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.090688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-37c3075b-0691-421e-ab6e-f21abe6449fd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.231527 Jul 20 14:52:49.091311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 322/641] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:49.367550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:49.368154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:49.375071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.375555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.376340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.376773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.529081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.537666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-37bb7006-b05e-4e14-aeb2-4b53c70c578d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.173908 Jul 20 14:52:49.538248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 320/642] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:49.613192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:49.614470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:49.615320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:49.616127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:49.621752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.622415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.622701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.623399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.623663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.624268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.624441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.624808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.842046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.848089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:49.854628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ee40e83a-8e37-4659-8752-934336d0a83e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.243344 Jul 20 14:52:49.855737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 321/643] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:49.862374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-aa625f36-41ff-473a-bbc2-39f23122c5fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.253139 Jul 20 14:52:49.863268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 323/644] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:49 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:49.882369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-6845f443-4d46-4ca2-ae52-1749ba81b008 req-5c157338-a1ee-4213-9cb4-958cc78f548f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "name": "network-vif-plugged", "status": "completed", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:49.893605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6845f443-4d46-4ca2-ae52-1749ba81b008 req-5c157338-a1ee-4213-9cb4-958cc78f548f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:49.894468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6845f443-4d46-4ca2-ae52-1749ba81b008 req-5c157338-a1ee-4213-9cb4-958cc78f548f service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:49.908485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-6845f443-4d46-4ca2-ae52-1749ba81b008 req-5c157338-a1ee-4213-9cb4-958cc78f548f service nova] Creating event network-vif-plugged:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:52:49.915334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-6845f443-4d46-4ca2-ae52-1749ba81b008 req-5c157338-a1ee-4213-9cb4-958cc78f548f service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035913 Jul 20 14:52:49.916309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 322/645] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:50.073026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:50.074244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:50.081926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.082379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.083220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.083656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.088237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:50.110171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:50.110870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:50.117615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.118068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.118842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.119325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.275201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:50.283658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-01b638f3-47ea-4c5a-b7a1-c276aaed591f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1555 microversion: 2.1 time: 0.215546 Jul 20 14:52:50.284249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 324/646] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:50 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1555 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:50.300830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:50.301210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:50.303423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:50.308004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.308436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.309211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.309639 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.312035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:50.313732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:50.314895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebc6f591-4beb-47f7-a076-6b3ee346b87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.209434 Jul 20 14:52:50.315482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 323/647] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:50.396296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ae481686-6eea-4c68-824a-b462a50aac17 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.099363 Jul 20 14:52:50.396898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 325/648] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 14:52:50 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/os-interface => generated 285 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:50.555568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:50.556180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:50.562612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.563092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.563854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.564298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.880750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:50.881366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:50.888307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.888756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:50.889537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:50.890003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.152121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.157780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=2 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:51.161830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-29fa4826-8e36-4ae6-aa71-bcf28ac1b832 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.609842 Jul 20 14:52:51.162783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 324/649] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:51.167859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:51.168471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:51.177000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.177616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:51.178073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.178853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:51.179342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.186202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a723a9a7-22ba-45b9-99ae-a3e930ba143d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.309770 Jul 20 14:52:51.195439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 326/650] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:50 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:51.335680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:51.336295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:51.338215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell0:SELECT=8 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:51.344098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:51.344547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.345337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:51.345771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.412626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.420434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0f1bcac-81c6-4149-8faa-03418395b542 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.256535 Jul 20 14:52:51.421028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 325/651] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:51 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:51.615707 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.624043 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.625724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:51.626917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-68d79c9d-864c-4afe-9724-6cb1392e9562 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.295274 Jul 20 14:52:51.627520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 327/652] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:51.953513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-6e966ab1-1d94-4696-8518-62e51453a14d req-7b7f0e57-02c9-42fe-87ce-665ac08a178d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88eb690a-bcac-4a8e-82df-86f0d0280558", "tag": "288d0f95-2b49-4578-b851-0f58b382c611"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:51.962239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6e966ab1-1d94-4696-8518-62e51453a14d req-7b7f0e57-02c9-42fe-87ce-665ac08a178d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:51.962680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-6e966ab1-1d94-4696-8518-62e51453a14d req-7b7f0e57-02c9-42fe-87ce-665ac08a178d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:51.974446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-6e966ab1-1d94-4696-8518-62e51453a14d req-7b7f0e57-02c9-42fe-87ce-665ac08a178d service nova] Creating event network-changed:288d0f95-2b49-4578-b851-0f58b382c611 for instance 88eb690a-bcac-4a8e-82df-86f0d0280558 on hv-cinder-84902 Jul 20 14:52:51.979300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-6e966ab1-1d94-4696-8518-62e51453a14d req-7b7f0e57-02c9-42fe-87ce-665ac08a178d service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029556 Jul 20 14:52:51.980111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 326/653] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:52.180549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:52.181125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:52.188302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.188847 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.189619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.190046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.203704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:52.204392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:52.211227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.211670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.212443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.212853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.229302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:52.400763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.409349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-697807f9-c197-4deb-8127-c838d0debf9c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.232734 Jul 20 14:52:52.410301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 328/654] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:52.417742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.427861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c22695c0-623d-4b3f-b41c-5da75417d6c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.228373 Jul 20 14:52:52.428421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 327/655] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:52.437783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:52.438397 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:52.444596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.446794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.446794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.446794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.584148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.592802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b0eb52c9-935f-4101-aefe-faa4b5845508 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.159190 Jul 20 14:52:52.593391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 329/656] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:52 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:52.645498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:52.646110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:52.648009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=20 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:52.653966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.654557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.655370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:52.655807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:52.809717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.822040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.823577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:52.824771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-472fab89-6c56-4ba9-a010-1f82193a89c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.183234 Jul 20 14:52:52.825677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 328/657] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:53.426779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:53.427400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:53.430301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:53.436381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.436831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.437591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.438047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.444972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:53.445597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:53.452837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.453342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.454072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.454518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.458912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:52:53.627798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:53.636590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b2948641-78de-4ae5-84e2-50864bf8c9e5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.213726 Jul 20 14:52:53.637175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 330/658] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:53 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:53.642263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:53.642867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:53.650324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.650752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.651585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.652017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.660660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:53.668381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cd65189d-0f73-4047-8fc0-ab7ad9cb9fb7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.227152 Jul 20 14:52:53.668969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 329/659] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:53.792373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:53.806221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cc7e39bd-5242-422e-9a58-c88ce9a70f4c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.167635 Jul 20 14:52:53.807131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 331/660] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:53.843809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:53.844424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:53.851298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.851756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.852546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:53.852985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:53.997129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:54.005920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:54.007437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:54.008666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b3950100-02ec-4248-bbc6-1b10de5eb7af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1745 microversion: 2.20 time: 0.169185 Jul 20 14:52:54.009196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 330/661] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1745 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:54.656942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:54.657567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:54.664666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.665103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.665868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.666296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.686788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:54.687419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:54.696980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.697264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.697977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.698421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.933104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:54.944126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-409a5e99-fb6e-44be-9bd1-b6ad1f849c52 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.292361 Jul 20 14:52:54.954438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 332/662] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:54 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:54.959623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:54.960328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:54.967966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.968519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.969420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:54.969959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:54.970790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:54.979855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c9f162cd-6815-4924-b1c1-369bfa8085f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.297731 Jul 20 14:52:54.980643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 331/663] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:54 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:55.027935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:55.028667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:55.035209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:55.035738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:55.040886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:55.040886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:55.168992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:55.180774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-88830450-cd17-4658-9929-aa65640a14e9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.224780 Jul 20 14:52:55.181664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 333/664] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:54 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:55.228098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:55.237229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:55.238679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:55.240001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-003c4266-fd44-457b-82b6-b690ececd314 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.216264 Jul 20 14:52:55.240552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 332/665] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:55.962001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:55.962597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:55.969920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:55.970347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:55.971388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:55.971982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:55.997043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:55.997873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:56.004473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.005184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.006132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.006721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.204892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.209724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.214339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-42b60bd1-8ad2-4f7c-9654-7abbca66d38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.256284 Jul 20 14:52:56.224253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfd7a116-ca0d-4b0b-a826-53886c14ebcb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.229075 Jul 20 14:52:56.224253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 333/666] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:55 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:56.224984 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 334/667] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:56.233077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:56.235793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:56.242480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.243227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.244177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.245092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.259543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:56.260615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:56.269166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.269756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.270562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.271002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.493859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.505785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-311a2d77-7682-4a82-97f5-2151b2ed5b2c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.276481 Jul 20 14:52:56.506489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 334/668] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:56.523474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.535850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.537757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:56.539163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-18b2bf3c-0ed1-4109-8a1f-32c6dd81f1d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.283970 Jul 20 14:52:56.540319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 335/669] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:56.573035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-f1a453ed-e0c0-43bd-b488-55160c0a1f90 req-37d7c0eb-83a6-46d7-b821-53e73408e0b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e827c810-220b-4fe1-9d16-01bc8372b933", "name": "network-vif-plugged", "status": "completed", "tag": "1f670f13-f9d2-4b6a-b0ce-7144e9adc59f"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:52:56.583349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f1a453ed-e0c0-43bd-b488-55160c0a1f90 req-37d7c0eb-83a6-46d7-b821-53e73408e0b1 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:56.583349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f1a453ed-e0c0-43bd-b488-55160c0a1f90 req-37d7c0eb-83a6-46d7-b821-53e73408e0b1 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:56.596108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-f1a453ed-e0c0-43bd-b488-55160c0a1f90 req-37d7c0eb-83a6-46d7-b821-53e73408e0b1 service nova] Creating event network-vif-plugged:1f670f13-f9d2-4b6a-b0ce-7144e9adc59f for instance e827c810-220b-4fe1-9d16-01bc8372b933 on hv-cinder-84902 Jul 20 14:52:56.601308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-f1a453ed-e0c0-43bd-b488-55160c0a1f90 req-37d7c0eb-83a6-46d7-b821-53e73408e0b1 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031898 Jul 20 14:52:56.602270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 335/670] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:52:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:52:57.231677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:57.233018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:57.240480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.240929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.241792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.242104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:57.242548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.243578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:57.253392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.254446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.255392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.256105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.462954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.471632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-92453e55-990c-45f6-988f-12dcf83954fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.243945 Jul 20 14:52:57.485045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.495112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 336/671] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:57.496253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-074a2fc0-e9f8-4007-9c9f-a80b7584b1e3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.259912 Jul 20 14:52:57.497051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 336/672] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:57.524104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:57.525017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:57.533280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.533779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.534605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.535148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.558347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:57.559428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:57.570971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.571820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.572869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:57.573495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:57.804686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.808432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.820754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.820754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:57.820754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb939a70-489b-4a65-b501-b60db215e53f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.266983 Jul 20 14:52:57.821674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 337/673] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:57 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:52:57.823992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-66fb026c-1e87-4364-9216-e4d3906d100a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.304341 Jul 20 14:52:57.829026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 337/674] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:57 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:58.492709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:58.494859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:58.502299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.502927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.503898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.504394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.513688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:58.514260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:58.525646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.525646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.525646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.525646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.737163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:58.748596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fbfa2f0f-4298-46c6-ad0a-faf4e9c76d13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.262686 Jul 20 14:52:58.760601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 338/675] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:58.782849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:58.796278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7814774e-d99d-4671-ada6-2f23c9edc58c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.285140 Jul 20 14:52:58.797280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 338/676] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:58 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:52:58.840131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:58.845740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:58.851793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:58.852519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:58.853321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.853951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.855112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.855729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.859695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.860239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:58.863955 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:58.864745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:59.094758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:59.106416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:59.108596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:59.110200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-536a1ea9-3bdd-4021-8680-837034cf9f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.274131 Jul 20 14:52:59.124015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 339/677] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:52:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:52:59.136322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:59.161662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c311b8ef-7e66-43e7-a5c7-e951f54f1b78 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1319 microversion: 2.1 time: 0.315898 Jul 20 14:52:59.162409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 339/678] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:58 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1319 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:52:59.766229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:59.767137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:59.774728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:59.778693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:59.779628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:59.780140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:59.823734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:52:59.826891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:52:59.834705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:59.836127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:59.837069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:52:59.837770 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:52:59.970000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:52:59.980014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe713faf-bae3-4ed0-acb6-d9f109f04ed4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1725 microversion: 2.1 time: 0.217803 Jul 20 14:52:59.991967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 340/679] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1725 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:00.027696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:00.038712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e09e3050-2d65-4f97-9e47-589051e00ea9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.227396 Jul 20 14:53:00.039446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 340/680] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:52:59 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:00.131147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:00.131757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:00.141006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:00.141451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:00.142230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:00.142732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:00.179018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:00.179656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:00.190332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:00.190773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:00.192021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:00.192476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:00.197084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:00.356419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:00.366682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:00.368480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:00.369677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4f9c7a4-2968-4265-a584-c857e39f7254 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.242553 Jul 20 14:53:00.381831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 341/681] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:00.408523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:00.422132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ffdcf3f9-1e01-4864-8590-d6f3a38ead4b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1508 microversion: 2.1 time: 0.246020 Jul 20 14:53:00.422132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 341/682] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1508 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:00.998195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:00.999535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:01.008907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.010007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.011071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.011492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.059124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:01.060453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:01.072466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.075341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.076356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.076795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.220234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.236680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3e0a3e75-4031-4d69-9f87-d5ed0bc513eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1702 microversion: 2.1 time: 0.241066 Jul 20 14:53:01.239802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 342/683] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1702 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:01.260440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1dd003b2-cbfd-4f68-8a68-30029c3f365e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:01.268902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1dd003b2-cbfd-4f68-8a68-30029c3f365e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.270255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1dd003b2-cbfd-4f68-8a68-30029c3f365e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.300943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1dd003b2-cbfd-4f68-8a68-30029c3f365e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Going to try to start instance {{(pid=124396) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 20 14:53:01.313267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.328395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5a6bc418-cd7a-4467-8a2d-d97db9e78a0e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.276078 Jul 20 14:53:01.330750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 342/684] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:01 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:01.388561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:01.389045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:01.395620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.396060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.396846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.397269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.398110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1dd003b2-cbfd-4f68-8a68-30029c3f365e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action" status: 202 len: 0 microversion: 2.1 time: 0.143531 Jul 20 14:53:01.399112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 343/685] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:53:01 2022] POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action => generated 0 bytes in 145 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:53:01.408262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:01.409125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:01.427818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.428474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.430274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:01.430880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:01.663714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.677227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.679839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.682369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4a2f8173-1036-438c-8125-099574d996a8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.297850 Jul 20 14:53:01.683142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 343/686] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:01.689135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:01.700521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d1c9cfd-39ae-42f2-89be-0573f591d88c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1711 microversion: 2.1 time: 0.296014 Jul 20 14:53:01.701600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 344/687] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1711 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:02.354320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:02.354951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:02.361732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.362180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.362952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.363437 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.561739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:02.573748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f95cc24a-1198-4b79-a834-535fde02a618 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.223144 Jul 20 14:53:02.584578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 344/688] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:02 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:02.701986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:02.703462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:02.712039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.713050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.714044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.714657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.722435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:02.723400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:02.739298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.740127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.741718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:02.741853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:02.742447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:02.939780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:02.949936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:02.951798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:02.953504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc2d4922-5c03-4410-a360-64bd52b97eff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.256327 Jul 20 14:53:02.958421 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:02.965325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 345/689] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:02.966916 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1f4a52f-f9d0-4958-b417-8ddca198fe3a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1711 microversion: 2.1 time: 0.252641 Jul 20 14:53:02.967521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 345/690] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1711 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:03.591849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:03.592460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:03.599083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.599550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:03.600322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.600790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:03.770245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:03.778362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-28a36c82-88a5-4906-9455-061a7869ab74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.190464 Jul 20 14:53:03.779105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 346/691] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:03 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:03.972556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:03.973164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:03.979814 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.980291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:03.981089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.981517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:03.985202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:03.985814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:03.992332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.992996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:03.993563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:03.994068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:04.202657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:04.213466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:04.214956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:04.216352 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd746263-1e1d-47e9-8855-0e0bc2041185 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.247855 Jul 20 14:53:04.217117 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 346/692] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:04.217950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:04.232006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61e9d39e-ebd2-400f-8264-c433ca8ac156 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1711 microversion: 2.1 time: 0.251403 Jul 20 14:53:04.233407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 347/693] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1711 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:04.796925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:04.797598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:04.805774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:04.806335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:04.807367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:04.808894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:04.996706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:05.007797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ffaccddb-ab90-4fab-a4b4-3f73365a2e1f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.215650 Jul 20 14:53:05.008735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 347/694] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:04 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:05.250209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:05.251238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:05.262929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:05.263704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:05.264648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:05.265250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:05.396614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:05.403450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:05.427806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:05.430752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:05.431668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:05.432186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:05.432969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:05.444145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-70f33ab9-7e1e-4772-9497-36378235dee1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1715 microversion: 2.1 time: 0.197673 Jul 20 14:53:05.445131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 348/695] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1715 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:05.624324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:05.633088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:05.634556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:05.635777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fa1245b9-cee4-45db-bcd2-ef6cd045aa93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.403236 Jul 20 14:53:05.646843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 348/696] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:06.027143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:06.027694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:06.034610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:06.035174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:06.035868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:06.036296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:06.191315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:06.199758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-85ef30b6-03c5-4aed-91d6-78f67ae04929 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.176842 Jul 20 14:53:06.200460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 349/697] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:06 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:06.655408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:06.656012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:06.662688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:06.663155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:06.663914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:06.664342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:06.800193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:06.808656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:06.810130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:06.811421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0589eec4-c571-4675-8f36-2d99c10790b1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1740 microversion: 2.20 time: 0.160882 Jul 20 14:53:06.811955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 349/698] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1740 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:07.217532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:07.218826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:07.226347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:07.226795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:07.227593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:07.228051 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:07.392238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:07.400772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-59bb2ba7-c821-4a69-ab25-daf5fd72be73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.187775 Jul 20 14:53:07.401373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 350/699] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:07 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:07.737505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-06e9692e-03eb-47cd-aa6b-7be63befcaee req-e261c438-eaf8-4354-beca-951a2dca686d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "name": "network-vif-plugged", "status": "completed", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:07.747894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-06e9692e-03eb-47cd-aa6b-7be63befcaee req-e261c438-eaf8-4354-beca-951a2dca686d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:07.748372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-06e9692e-03eb-47cd-aa6b-7be63befcaee req-e261c438-eaf8-4354-beca-951a2dca686d service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:07.762476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-06e9692e-03eb-47cd-aa6b-7be63befcaee req-e261c438-eaf8-4354-beca-951a2dca686d service nova] Creating event network-vif-plugged:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 14:53:07.768692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-06e9692e-03eb-47cd-aa6b-7be63befcaee req-e261c438-eaf8-4354-beca-951a2dca686d service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035395 Jul 20 14:53:07.770039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 350/700] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:53:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:07.830202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:07.831204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:07.837594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:07.838035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:07.838797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:07.839274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:08.149956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:53:08.150243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 14:53:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d (ip 10.222.0.15) !!! Jul 20 14:53:08.244794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:08.253719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:08.255949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:08.257223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d03eac4e-48ec-4dde-89b7-ac9d3a3b1cdb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1722 microversion: 2.20 time: 0.431496 Jul 20 14:53:08.257781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 351/701] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1722 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:08.419469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:08.420072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:08.427808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:08.428522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:08.429779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:08.430375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:08.578558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:08.586553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c7020ced-7406-413b-b0a1-b592b9986a53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.171837 Jul 20 14:53:08.587158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 351/702] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:08 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:09.276938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:09.277549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:09.284951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:09.285404 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:09.286164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:09.286585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:09.432961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:09.441435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:09.442982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:09.444170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a55b3873-1a69-4d0f-9b62-aca6cc804a12 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1722 microversion: 2.20 time: 0.172145 Jul 20 14:53:09.444716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 352/703] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1722 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:09.607384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:09.607968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:09.615002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:09.615547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:09.616285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:09.616706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:09.755033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:09.765108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a90d6a67-54cd-4932-82f5-a4d52596fe13 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.161294 Jul 20 14:53:09.765663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 352/704] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:09 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:10.462839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:10.463652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:10.470316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:10.470757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:10.471554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:10.472125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:10.616959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:10.625953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:10.627501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:10.628706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-10f035c5-b4d8-4b2a-b4e2-536852b6244b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1722 microversion: 2.20 time: 0.170072 Jul 20 14:53:10.629266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 353/705] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1722 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:10.784641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:10.785303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:10.793649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:10.794484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:10.795566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:10.796355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:10.938165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:10.948834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e3c40ce5-f962-405f-a169-46289b4155ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.168685 Jul 20 14:53:10.949550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 353/706] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:10 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:11.650568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:11.652475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:11.662399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.662743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.663656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-7395e243-4948-4dc8-ac81-2b99c4158072 req-ed5c97c8-8e43-49c0-96d4-1e4b0fc5e9aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "name": "network-vif-unplugged", "status": "completed", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:11.663815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.664024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.672584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7395e243-4948-4dc8-ac81-2b99c4158072 req-ed5c97c8-8e43-49c0-96d4-1e4b0fc5e9aa service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.673028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7395e243-4948-4dc8-ac81-2b99c4158072 req-ed5c97c8-8e43-49c0-96d4-1e4b0fc5e9aa service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.690494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-7395e243-4948-4dc8-ac81-2b99c4158072 req-ed5c97c8-8e43-49c0-96d4-1e4b0fc5e9aa service nova] Creating event network-vif-unplugged:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:53:11.695640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-7395e243-4948-4dc8-ac81-2b99c4158072 req-ed5c97c8-8e43-49c0-96d4-1e4b0fc5e9aa service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036120 Jul 20 14:53:11.696230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 354/707] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:53:11 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:11.808809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:11.817229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:11.818710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:11.819905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6cfaba37-d1e2-4b14-a644-45a1936d8e0d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1725 microversion: 2.20 time: 0.175904 Jul 20 14:53:11.820490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 354/708] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1725 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:11.839415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44b5537a-5392-4a04-990f-113d3e14c717"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:11.840557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:11.847378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.847880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.848642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.849068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.973931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:11.975030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:11.985715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.986360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:11.987543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:11.988280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:12.160306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:12.165380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-92ff0c7a-1945-48f1-b1fb-daf39ebc02c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.198176 Jul 20 14:53:12.165989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 355/709] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:11 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:12.187324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-83bda5ba-b10c-4d78-8b5e-1b185c07ac54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.352368 Jul 20 14:53:12.187944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 355/710] 10.222.0.15 () {62 vars in 1363 bytes} [Wed Jul 20 14:53:11 2022] POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 188 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 14:53:12.316922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:12.317806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:12.325398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:12.326205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:12.327590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:12.328088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:12.481039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:12.499791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c282aef4-45e0-44fe-a7f0-cbca6d0731dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action" status: 202 len: 0 microversion: 2.20 time: 0.187290 Jul 20 14:53:12.500837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 356/711] 10.222.0.15 () {62 vars in 1318 bytes} [Wed Jul 20 14:53:12 2022] POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:53:12.510709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:12.511650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:12.518620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:12.519085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:12.519896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:12.520330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:12.672638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:12.681203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:12.682656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:12.683838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bd6c2255-015e-4cb7-8de4-cbe5ad3ce2cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1751 microversion: 2.20 time: 0.178493 Jul 20 14:53:12.684652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 356/712] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1751 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:13.182125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:13.182792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:13.194941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:13.195409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:13.196178 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:13.196604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:13.342563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:13.352081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-46610475-b10f-4241-b275-c2400af64277 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1261 microversion: 2.1 time: 0.174107 Jul 20 14:53:13.357974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 357/713] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:13 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1261 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:13.702103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:13.703029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:13.709678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:13.710389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:13.711374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:13.711968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:13.870949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:13.881368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:13.882837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:13.884018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e4ef9a15-110f-42d7-b00e-3222047c94dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1751 microversion: 2.20 time: 0.186039 Jul 20 14:53:13.884544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 357/714] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1751 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:14.369750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:14.370440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:14.377419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.377864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:14.378632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.379077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:14.528050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:14.536146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bed90052-82af-4808-95a2-cbb1f09c104c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1444 microversion: 2.1 time: 0.170225 Jul 20 14:53:14.536727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 358/715] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:14 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1444 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:14.552929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:14.553535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:14.559787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.560233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:14.560997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.561442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:14.702414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:14.710701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-89062195-1e2c-4dac-a58b-77ef5bd87366 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1444 microversion: 2.1 time: 0.161562 Jul 20 14:53:14.711350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 358/716] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:14 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1444 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:14.903215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:14.903957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:14.910576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.911183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:14.912072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:14.912535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:15.136464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:15.145647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:15.147286 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:15.148468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a3eb1809-0f12-4c6b-abbf-8a3998a7ebdc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1753 microversion: 2.20 time: 0.249614 Jul 20 14:53:15.149347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 359/717] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1753 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:16.167989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:16.168693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:16.175336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:16.175881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:16.176752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:16.177275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:16.432886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:16.442371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:16.444344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:16.445450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e88d10b0-61ce-4ee1-931a-92b42b35a645 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.281433 Jul 20 14:53:16.456056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 359/718] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:16.739555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-ad06ee87-792e-4353-8bd3-140f4ad55cd4 req-6b9d673c-07f7-4298-9314-f9b521ec3c18 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:16.749074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-ad06ee87-792e-4353-8bd3-140f4ad55cd4 req-6b9d673c-07f7-4298-9314-f9b521ec3c18 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:16.749508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-ad06ee87-792e-4353-8bd3-140f4ad55cd4 req-6b9d673c-07f7-4298-9314-f9b521ec3c18 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:16.761596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-ad06ee87-792e-4353-8bd3-140f4ad55cd4 req-6b9d673c-07f7-4298-9314-f9b521ec3c18 service nova] Creating event network-changed:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 14:53:16.766861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-ad06ee87-792e-4353-8bd3-140f4ad55cd4 req-6b9d673c-07f7-4298-9314-f9b521ec3c18 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030906 Jul 20 14:53:16.767853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 360/719] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:53:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:17.464497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:17.465115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:17.472206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:17.472698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:17.473557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:17.474015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:17.708675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:17.712622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:17.719162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:17.720986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:17.722514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-975e4f25-343d-4e79-ab9e-104874e63442 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.262229 Jul 20 14:53:17.723750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 360/720] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:53:18.742769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:18.743366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:18.750162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:18.750605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:18.751425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:18.751840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:18.901251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:18.911253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:18.913189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:18.914642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4c7f064c-e9a4-41ec-9b16-3ccc833c2894 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.176765 Jul 20 14:53:18.915258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 361/721] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:19.933280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:19.933980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:19.938212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=89 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:19.942182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:19.942589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:19.943633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:19.944247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:20.085295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:20.095183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:20.096691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:20.097866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dafafe6f-c61b-4abe-9ed2-f34eaf5795e8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.168515 Jul 20 14:53:20.098690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 361/722] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:21.120709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:21.121378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:21.128713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:21.129130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:21.129893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:21.130319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:21.275785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:21.292083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:21.293755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:21.295029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-abc3572d-fdde-4492-b571-28910f8addb2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.178395 Jul 20 14:53:21.295629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 362/723] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:22.314108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:22.314777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:22.321404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:22.321778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:22.322513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:22.323180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:22.460187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:22.469188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:22.470666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:22.471878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e826373f-aba4-4de1-813a-9f6cc3e07995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.161870 Jul 20 14:53:22.472433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 362/724] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:23.490570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:23.491212 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:23.493652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=68 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:23.499265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:23.499723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:23.500483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:23.500915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:23.664412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:23.673342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:23.674801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:23.675987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d0bbfa9-1bff-4f66-adb9-e90f58ef2107 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.189347 Jul 20 14:53:23.676547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 363/725] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:24.694769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:24.695514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:24.702739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:24.702739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:24.703544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:24.703972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:24.857762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:24.866744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:24.868272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:24.869438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92db82e5-b135-4d57-86bf-38632fc27122 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.178876 Jul 20 14:53:24.869997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 363/726] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:25.888812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:25.889553 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:25.896526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:25.896981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:25.897768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:25.898203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:26.038592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:26.048024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:26.049516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:26.050686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-39ff6fcc-12dd-4e5f-a692-6f092f23c554 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.165980 Jul 20 14:53:26.051274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 364/727] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:27.068749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:27.069414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:27.075765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:27.076209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:27.076980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:27.079598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:27.224327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:27.233674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:27.236028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:27.237699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-829d499a-2098-4549-bee7-644ec7057512 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.172967 Jul 20 14:53:27.238770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 364/728] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:27.998401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "66cea3b7-c23c-43b5-ba14-cdcdf0c9952b"}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:27.999558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:28.005792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:28.006235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:28.006996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:28.007446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:28.260651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:28.261625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:28.271869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:28.272516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:28.274098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:28.274816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:28.420682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:28.430554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:28.432284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:28.433663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4ff31ff0-84b8-40af-ad4e-4527c235c5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.180204 Jul 20 14:53:28.439918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 365/729] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:29.452754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:29.453447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:29.459883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:29.460307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:29.461106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:29.461529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:29.613673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:29.627837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:29.629321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:29.630530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-525b9b10-4ad2-40a7-a5f0-0e0af355b916 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.182415 Jul 20 14:53:29.631095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 366/730] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:29.635985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "eaa89bcc-652b-4a03-941b-8440920143e6", "device": "/dev/sdb"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:29.637735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:29.644567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:29.645008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:29.645827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:29.646257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:35.209336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a086f5d3-0205-444a-824f-ee961df6ce72 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "POST /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.576377 Jul 20 14:53:35.210660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 367/731] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:53:29 2022] POST /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments => generated 194 bytes in 5579 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:35.211099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2a12fd54-fd24-4a04-ba94-479defa67984 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 7.341815 Jul 20 14:53:35.212212 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 365/732] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:53:27 2022] POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 194 bytes in 7344 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:35.216468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:35.217171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:35.224194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:35.224571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:35.225348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:35.225784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:35.369314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:35.379335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:35.380818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:35.381994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0169f53d-eaea-4f24-8314-d6c6083d8afa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.169732 Jul 20 14:53:35.382535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 368/733] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:36.401697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:36.402389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:36.409832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:36.410305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:36.411097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:36.411556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:36.554845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:36.563982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:36.565506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:36.566716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-607c5bdf-cd8d-466a-a5ec-31c0ae967e9c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.170179 Jul 20 14:53:36.567359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 366/734] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:37.585430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:37.586004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:37.593174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:37.593623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:37.594392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:37.594823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:37.738011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:37.746797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:37.748326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:37.749500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a39dee25-baef-4bc5-b0fe-4b14c5ed88c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.168119 Jul 20 14:53:37.750081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 369/735] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:38.768428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:38.769004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:38.776327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:38.776812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:38.777582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:38.778014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:38.931834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:38.942162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:38.943717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:38.944920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7b51daff-d084-48fd-ba9f-7cc62a32d523 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.180704 Jul 20 14:53:38.945469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 367/736] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:39.963917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:39.964493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:39.971582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:39.972030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:39.972785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:39.973244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:40.114282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:40.126441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:40.128700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:40.130499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40149d1e-3f19-4327-9eb5-9c9f3f5f7756 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.170504 Jul 20 14:53:40.131551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 370/737] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:41.153426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:41.154600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:41.161820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:41.162264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:41.163041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:41.163508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:41.303872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:41.313704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:41.315223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:41.316466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-93dfffa9-85cd-447e-897b-fdeee6fd947f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.169512 Jul 20 14:53:41.316998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 368/738] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:42.336418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:42.337284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:42.343836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:42.344284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:42.345048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:42.345479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:42.488237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:42.497596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:42.499188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:42.500353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c44a54ea-c7b0-432e-8d95-1a2efd8cd9f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.168586 Jul 20 14:53:42.500914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 371/739] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:43.518801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:43.519425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:43.527879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:43.528323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:43.529231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:43.529749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:43.532729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:43.673081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:43.681968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:43.683486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:43.684661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fbd4a985-ecc2-4b69-8699-de23f6dd3074 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.169927 Jul 20 14:53:43.685260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 369/740] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:44.703291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:44.703914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:44.710945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:44.711418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:44.712196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:44.712690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:44.847712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:44.856095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:44.857575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:44.858730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f6b6a4ac-7fc6-4b83-bd09-12cc9b033178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.159605 Jul 20 14:53:44.859290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 372/741] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:45.876679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:45.877184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:45.885863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:45.886414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:45.887263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:45.887698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:45.958992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:45.959575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:45.970703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:45.971925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:45.973196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:45.973871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:46.005788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:53:46.077364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:46.087884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:46.089048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:46.090269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b8f2d5aa-4105-4827-92c0-9e87c42d759c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.218192 Jul 20 14:53:46.090838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 370/742] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:46.103499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4f48c659-39c1-4171-a975-6c6b7eb51b38 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 204 len: 0 microversion: 2.1 time: 0.148157 Jul 20 14:53:46.104218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 373/743] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:53:45 2022] DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 20 14:53:46.119790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:46.120750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:46.129949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:46.130664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:46.131480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:46.131918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:46.274026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:46.285819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9e2e4198-189a-436e-b0d3-59ea61d85529 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.177315 Jul 20 14:53:46.286479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 371/744] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:46 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:47.109462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:47.110314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:47.117488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:47.117893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:47.118655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:47.119095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:47.140609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:47.268265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:47.277177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:47.278642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:47.279896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61c9fd3f-9ff0-4526-b574-fd2b0a8b81e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.174830 Jul 20 14:53:47.280458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 374/745] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:47.303833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:47.304447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:47.311652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:47.312081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:47.312843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:47.313302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:47.451273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:47.459512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-85bff8a8-5564-46f7-9e10-6ba3da1ee52c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.159530 Jul 20 14:53:47.460030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 372/746] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:47 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:48.299502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:48.300093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:48.307067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:48.307523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:48.308297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:48.308732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:48.458753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:48.467750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:48.469242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:48.470477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d597d6da-f2ab-4e11-95c1-c6b7b2e8826a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.175160 Jul 20 14:53:48.471026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 375/747] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:48.477071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:48.477685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:48.485851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:48.486314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:48.487088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:48.487789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:48.610151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-41dbac88-017a-4f7d-ada6-643602bf0f3d req-dc21a93d-14b1-424e-b971-59ee487064f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "name": "network-vif-plugged", "status": "completed", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:48.619227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-41dbac88-017a-4f7d-ada6-643602bf0f3d req-dc21a93d-14b1-424e-b971-59ee487064f1 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:48.620337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-41dbac88-017a-4f7d-ada6-643602bf0f3d req-dc21a93d-14b1-424e-b971-59ee487064f1 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:48.632189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:48.632798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-41dbac88-017a-4f7d-ada6-643602bf0f3d req-dc21a93d-14b1-424e-b971-59ee487064f1 service nova] Creating event network-vif-plugged:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 14:53:48.637903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-41dbac88-017a-4f7d-ada6-643602bf0f3d req-dc21a93d-14b1-424e-b971-59ee487064f1 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031390 Jul 20 14:53:48.638589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 376/748] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:53:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:48.641606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7fd8a422-b38b-447a-bc13-319018476c4e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.168625 Jul 20 14:53:48.642170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 373/749] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:48 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:49.494749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:49.494749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:49.499339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:49.499816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:49.500590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:49.501025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:49.638029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:49.647184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:49.648981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:49.650168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2c25572e-e2fb-46f7-b8cd-fd25cebb575f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.164960 Jul 20 14:53:49.650735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 377/750] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:49.659134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:49.660380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:49.668133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:49.668580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:49.669391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:49.669830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:49.809748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:49.817584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dc822e30-f698-47bd-8540-d2e8219fdf98 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.162276 Jul 20 14:53:49.818163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 374/751] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:49 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:50.669267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:50.669832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:50.672523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:50.678508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:50.678950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:50.679751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:50.680186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:50.834742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:50.835079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:50.835663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:50.842522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:50.842992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:50.843832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:50.843953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:50.844268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:50.845349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:50.846520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc07778a-550c-4649-80b5-e4faeeaeb087 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.181368 Jul 20 14:53:50.847302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 378/752] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:50.986900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:50.994864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a72697ea-06ea-4439-9770-10dddc1aa9e5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.163677 Jul 20 14:53:50.995486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 375/753] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:50 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:51.870164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:51.871294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:51.880553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:51.881315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:51.882511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:51.883237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:52.012844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:52.013430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:52.020530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:52.020982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:52.021766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:52.022223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:52.036392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:52.045510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:52.046831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:52.048045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a62889de-9319-4709-b60f-637ab426b75c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.185043 Jul 20 14:53:52.048612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 379/754] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:52.166688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:52.176587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fe924e68-a867-40a3-9681-3d23826dd196 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.168067 Jul 20 14:53:52.177173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 376/755] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:52 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:52.611201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-708eec22-50b1-48e0-9ac2-b622cd1e3286 req-5ae2c8f0-23c6-4ee5-a2f4-ff7517714eec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f3521d1-81cb-4217-b43a-b84b94911839", "name": "network-vif-unplugged", "status": "completed", "tag": "ae0ff63e-d9ee-41cd-875c-523dfce9bd0c"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:53:52.619901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-708eec22-50b1-48e0-9ac2-b622cd1e3286 req-5ae2c8f0-23c6-4ee5-a2f4-ff7517714eec service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:52.620365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-708eec22-50b1-48e0-9ac2-b622cd1e3286 req-5ae2c8f0-23c6-4ee5-a2f4-ff7517714eec service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:52.634387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-708eec22-50b1-48e0-9ac2-b622cd1e3286 req-5ae2c8f0-23c6-4ee5-a2f4-ff7517714eec service nova] Creating event network-vif-unplugged:ae0ff63e-d9ee-41cd-875c-523dfce9bd0c for instance 2f3521d1-81cb-4217-b43a-b84b94911839 on hv-cinder-84902 Jul 20 14:53:52.639621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-708eec22-50b1-48e0-9ac2-b622cd1e3286 req-5ae2c8f0-23c6-4ee5-a2f4-ff7517714eec service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.111343 Jul 20 14:53:52.640219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 380/756] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:53:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:53:53.067228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:53.067845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:53.074901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:53.075385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:53.076151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:53.076584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:53.195269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:53.196005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:53.203234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:53.203702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:53.204508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:53.204951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:53.529903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:53.542149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:53.543750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:53.544966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ce446499-ed95-485b-a6e9-3ed6b86a7c2d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.481703 Jul 20 14:53:53.545562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 377/757] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:53.555514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:53.565295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-975024bc-f5b6-44ef-a6ba-765277ce9da9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.374930 Jul 20 14:53:53.565916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 381/758] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:54.563879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:54.564487 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:54.566816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=30 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:53:54.572581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:54.573030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:54.573795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:54.574204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:54.700743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:54.701463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:54.707899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:54.708344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:54.709114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:54.709549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:54.713472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:54.724590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:54.726311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:54.727839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-95f7c810-20c3-47a2-9f46-93a9f621b049 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.167813 Jul 20 14:53:54.729387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 378/759] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:54.859478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:54.867249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3c25f472-14f0-4805-b10c-ed1acb382d13 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.288913 Jul 20 14:53:54.867888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 382/760] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:55.746495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:55.747105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:55.754143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:55.754588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:55.755388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:55.755813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:55.884515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:55.885233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:55.888612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:55.892271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:55.892716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:55.893483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:55.893908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:55.897551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:55.899433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:55.900613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-91d06c48-c075-4608-9dae-fc5ab0899a1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.158172 Jul 20 14:53:55.901175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 379/761] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:56.030816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:56.038907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-69f1e40f-e78f-4ba0-b12d-d75a91770156 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.158434 Jul 20 14:53:56.039523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 383/762] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:55 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:56.922022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:56.922895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:56.932978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:56.933554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:56.934816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:56.935334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:57.061920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:57.063024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:57.070361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:57.070810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:57.071626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:57.072063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:57.076349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:57.085198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:57.086726 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:57.087940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-11bd692d-b310-40cc-a083-86f552d82c53 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.172384 Jul 20 14:53:57.088502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 380/763] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:57.211090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:57.219173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9433da19-f04c-48f8-b660-5ed10585063b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.165246 Jul 20 14:53:57.219739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 384/764] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:57 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:53:58.108928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:58.109577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:58.117659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:58.118101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:58.118874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:58.119339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:58.236443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:58.237148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:58.244475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:58.244946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:58.245695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:58.246129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:58.260531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:58.269358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:58.270857 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:58.272060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6f0557e9-35bf-4ee8-91c4-f7f4b91124ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.167716 Jul 20 14:53:58.272657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 381/765] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:58.388228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:58.396817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2ab84a1f-60e0-4343-a4de-c57b0a46d8b7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.164364 Jul 20 14:53:58.397402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 385/766] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:58 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:53:59.291530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:59.292104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:59.299080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:59.299567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:59.300734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:59.301454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:59.413827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:53:59.414533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:53:59.421449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:59.421997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:59.422846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:53:59.423409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:53:59.459198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:59.468188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:59.470202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:59.471399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-822b05c8-521b-4021-9873-1fcf25d90994 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.184193 Jul 20 14:53:59.472168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 382/767] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:53:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:53:59.569080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:53:59.577118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-28b3c128-3068-4ed7-a132-95ba511f4434 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.166941 Jul 20 14:53:59.577661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 386/768] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:53:59 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:00.494195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:00.494924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:00.502510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:00.502955 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:00.503780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:00.504210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:00.594293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:00.594960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:00.601654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:00.602079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:00.603209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:00.603894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:00.657066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:00.667389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:00.669830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:00.671792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1fc38964-c489-4448-8dc6-571d2a065b67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.182468 Jul 20 14:54:00.672372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 383/769] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:00.744416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:00.753361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4509e86e-bae7-4a4f-bc90-ee97bee81033 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.163154 Jul 20 14:54:00.753942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 387/770] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:00 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:01.690016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:01.690618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:01.697758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:01.698207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:01.698977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:01.699449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:01.770111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:01.770722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:01.777894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:01.778316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:01.779084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:01.779527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:01.867563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:01.876584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:01.878083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:01.879323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-60c5253a-7fef-4219-b869-9ad5b8d6aae2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.193229 Jul 20 14:54:01.879887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 384/771] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:01.934844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:01.942778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3e3a5cf5-7b17-424a-ae78-a6162faeb79c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.176544 Jul 20 14:54:01.943390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 388/772] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:01 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:02.900829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:02.901418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:02.909267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:02.909717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:02.910507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:02.910939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:02.960125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:02.960691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:02.967921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:02.968368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:02.969153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:02.969579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:03.094981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:03.094981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:03.102698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:03.104184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:03.105400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d93c31c9-40e6-48ec-9663-2c2c83ed0f70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.209015 Jul 20 14:54:03.105968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 385/773] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:03.140353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:03.149117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2b696759-e41b-4564-b6ef-df7ad37fc623 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.192829 Jul 20 14:54:03.149699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 389/774] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:02 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:04.125915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:04.126627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:04.136140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:04.136740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:04.137748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:04.138322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:04.171652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:04.172217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:04.179248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:04.179693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:04.180466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:04.180890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:04.334693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:04.345151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:04.346840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:04.348228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-db236637-ddd3-45d0-9d22-fcd1ae1f83f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.227660 Jul 20 14:54:04.348771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 386/775] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:04.364768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:04.373067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-54fa6f37-eb7a-4a3a-9c23-7b581e83ae33 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.205328 Jul 20 14:54:04.373679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 390/776] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:04 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:05.369359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:05.370160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:05.378415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:05.378859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:05.379643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:05.380068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:05.389800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:05.390370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:05.397983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:05.398436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:05.399259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:05.399696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:05.585203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:05.588817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:05.596335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:05.596560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3c219c0-a1f4-4d5d-8721-6177a551cc07 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.210693 Jul 20 14:54:05.597131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 391/777] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:05 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:05.598037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:05.599281 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e0e335ac-b01d-42aa-9aea-4a4de3d2e7b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.236145 Jul 20 14:54:05.599836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 387/778] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:06.614214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:06.614823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:06.617427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:06.618083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:06.623526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:06.624020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:06.624797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:06.624917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:06.625229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:06.625348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:06.626020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:06.626456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:06.644463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:06.829331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:06.832657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:06.837369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da04b123-a678-4825-a834-4c9c68df82aa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.226868 Jul 20 14:54:06.837890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 392/779] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:06 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:06.841838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:06.843416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:06.844587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7b1037b2-7847-40b8-80b2-0d4494e2a58b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.231034 Jul 20 14:54:06.845168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 388/780] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:07.857200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:07.858129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:07.863998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:07.864634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:07.867942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:07.868945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:07.869819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:07.870475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:07.870859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:07.871315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:07.872080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:07.872512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:08.074598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:08.078105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:08.084248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:08.085759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:08.086171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1584a7fc-292c-4980-80a3-3f3e0dfbdc36 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.235489 Jul 20 14:54:08.086741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 393/781] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:07 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:08.086942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5fc7c92a-a845-4710-a0a0-e38c9388ab8a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.227347 Jul 20 14:54:08.087516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 389/782] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:09.107744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:09.108576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:09.109187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:09.109600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:09.119649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:09.120406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:09.120649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:09.121126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:09.121740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:09.122495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:09.122745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:09.123283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:09.323889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:09.327756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:09.333015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-95cd0bb8-c52a-4aed-b3da-ef3e803fa6fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.231813 Jul 20 14:54:09.333588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 394/783] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:09 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:09.337447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:09.338928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:09.340152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b7b230bd-0cde-4685-b71f-6e0850474780 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.237926 Jul 20 14:54:09.340738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 390/784] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:10.351277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:10.351898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:10.359285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:10.359947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:10.361076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:10.361624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:10.362833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:10.363482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:10.369148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:10.369649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:10.370378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:10.370851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:10.572837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:10.578745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:10.581658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76311cb1-e262-4fcd-8697-7712c8b232b2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.235225 Jul 20 14:54:10.582217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 395/785] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:10 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:10.587976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:10.589478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:10.590736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-46e8d0ce-cb65-4ec0-bb58-2a16fa5c6bfb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.236832 Jul 20 14:54:10.591220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 391/786] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:11.599259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:11.599870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:11.607141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:11.607634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:11.609823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:11.609823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:11.612778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:11.613577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:11.620927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:11.621374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:11.622148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:11.622573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:11.817585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:11.829298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-79eee5e9-c412-46e8-b724-69f38bc28f80 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.233646 Jul 20 14:54:11.830069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 396/787] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:11 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:11.838724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:11.848168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:11.849689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:11.850811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2c4e8c43-1a72-4361-8227-809401f20661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.241997 Jul 20 14:54:11.851389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 392/788] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:12.848813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:12.849440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:12.856582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:12.857030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:12.857798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:12.858232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:12.871956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:12.872548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:12.881966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:12.882418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:12.883231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:12.883670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:13.072762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:13.076896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:13.081212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d18d4073-a8f0-4097-b094-575b4477ea79 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.236234 Jul 20 14:54:13.081776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 397/789] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:12 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:13.086899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:13.088469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:13.089657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c8ee5b7b-4a74-428f-913b-b8413ea376ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.223156 Jul 20 14:54:13.090214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 393/790] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:14.101077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:14.101723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:14.110695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:14.110987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:14.111486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:14.111598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:14.111974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:14.112378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:14.119624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:14.120077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:14.120857 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:14.121292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:14.322437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:14.331442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7dafcb3f-6f2b-4268-adc0-497ca8dc7f2f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.236181 Jul 20 14:54:14.332169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 398/791] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:14 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:14.338748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:14.347838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:14.349318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:14.350489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d8d60187-64a4-4c4e-99b6-41df194b4d7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.245524 Jul 20 14:54:14.351072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 394/792] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:15.351097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:15.351757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:15.358710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:15.359182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:15.359954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:15.360383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:15.370050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:15.370638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:15.377755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:15.378202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:15.378985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:15.379457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:15.581374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:15.584993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:15.589477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56fa58fb-83bf-4989-a401-6e7793a95c60 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.244025 Jul 20 14:54:15.590063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 399/793] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:15 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:15.596225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:15.597835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:15.599010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-20b73742-6506-4891-9b3a-06779c224677 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.232937 Jul 20 14:54:15.599572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 395/794] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:16.607232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:16.607830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:16.615183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:16.615638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:16.616407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:16.616812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:16.617366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:16.617965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:16.625754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:16.626351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:16.627232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:16.627664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:16.823278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:16.827082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:16.832252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b12425d-3220-44d5-a4f6-ce5b937d1bb9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.228993 Jul 20 14:54:16.832982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 400/795] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:16 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:16.836548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:16.838025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:16.839218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-55bafc55-ab07-4e93-9676-1e006193b566 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.225718 Jul 20 14:54:16.839956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 396/796] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:17.850967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:17.851640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:17.857754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:17.858419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:17.858916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:17.859388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:17.860170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:17.860599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:17.867172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:17.867648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:17.868427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:17.868869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:18.079960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:18.087418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:18.087864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-530c521b-1838-4e6a-8055-c1789159653a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.241772 Jul 20 14:54:18.088489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 401/797] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:17 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:18.097768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:18.099912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:18.101891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-21d3f8f7-aa63-4e97-b3c4-e4ba20d59943 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.247772 Jul 20 14:54:18.102539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 397/798] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:19.109558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:19.110636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:19.120205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:19.120811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:19.121709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:19.122147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:19.122576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:19.123249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:19.130396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:19.130844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:19.131668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:19.132100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:19.332280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:19.336616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:19.340683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f4367a7-f468-4f8f-b1f6-e89e2c2eabc9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.238091 Jul 20 14:54:19.341275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 402/799] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:19 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:19.346127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:19.347652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:19.348841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8c8c7cd0-2a09-4e67-97ab-4378bbf2e90e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.231731 Jul 20 14:54:19.349403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 398/800] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:20.358706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:20.359371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:20.366615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:20.366955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:20.367749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:20.368413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:20.368413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:20.368686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:20.375705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:20.376148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:20.376924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:20.377355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:20.571797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:20.573852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:20.579205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e6155855-683a-4f3e-9f3d-8f1f7ed6ba43 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.224274 Jul 20 14:54:20.579791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 403/801] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:20 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:20.583644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:20.585147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:20.586354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8235b44d-431a-4ecb-981c-f56170b39267 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.222528 Jul 20 14:54:20.586906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 399/802] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:21.596822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:21.597445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:21.599802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=52 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:21.604758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:21.605398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:21.605529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:21.605983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:21.606757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:21.607237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:21.612489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:21.613069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:21.613850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:21.614283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:21.814530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:21.818952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:21.822880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-15188d95-ce8b-40c3-8a51-8b6e6c3e9b6d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.229883 Jul 20 14:54:21.823505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 404/803] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:21 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:21.827850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:21.829344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:21.830616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1c3f0595-3fe2-416c-b6f1-f7f6d41d54db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.229705 Jul 20 14:54:21.831191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 400/804] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:22.840594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:22.841216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:22.848745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:22.849187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:22.849961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:22.850371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:22.850971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:22.851127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:22.859223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:22.859763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:22.862932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:22.862932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:23.082748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:23.091613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92290a93-b6ce-454b-95fc-87687fb1608d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.254488 Jul 20 14:54:23.092255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 405/805] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:22 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:23.097456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:23.106936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:23.108684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:23.109858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-812c1880-96bd-4624-b71e-69bbea258cd7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.263823 Jul 20 14:54:23.110415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 401/806] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:24.109646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:24.110400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:24.117536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:24.117980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:24.118756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:24.119218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:24.129317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:24.129926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:24.137151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:24.137603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:24.138399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:24.138927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:24.157164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:24.353140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:24.359733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:24.361919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16917b17-3480-4c86-9800-a439dfa42b53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.256048 Jul 20 14:54:24.362587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 406/807] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:24 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:24.368740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:24.370214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:24.371443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0dcb55f6-5dc5-4f98-8c59-35b640d7ca3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.246179 Jul 20 14:54:24.372007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 402/808] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:25.218798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:25.219664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:25.229608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.230050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.230820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.231372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.374183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'services' on Instance uuid 52220445-7cc1-4635-9566-c6fd58b1fc96 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:54:25.379926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:25.380545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:25.383908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=50 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:25.390499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.391371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.394371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.394371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.498813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2989b49-7690-42bd-86e0-045b6e63ae1e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments/647e4365-be89-4100-91f7-a1ee50e4487a" status: 202 len: 0 microversion: 2.1 time: 0.394713 Jul 20 14:54:25.499458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 407/809] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 14:54:25 2022] DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments/647e4365-be89-4100-91f7-a1ee50e4487a => generated 0 bytes in 396 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:54:25.505123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:25.505861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:25.512584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.513026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.513825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:25.514269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:25.534059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:25.535521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:25.544575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-73f88dd5-2c83-4ae5-bc5b-61d1660ce3b9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.169025 Jul 20 14:54:25.545259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 403/810] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:25 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:25.663679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:25.674774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:25.676306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:25.677465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-afa2ede2-d7c3-4deb-abc2-4df13b98244e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.176903 Jul 20 14:54:25.677998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 408/811] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:26.562183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:26.562801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:26.571627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:26.572366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:26.573634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:26.574356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:26.695935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:26.696714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:26.703604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:26.704055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:26.704833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:26.705264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:26.707809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:26.715505 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-237ee721-5b37-45de-8065-9521d94f4ea2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.157276 Jul 20 14:54:26.716149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 404/812] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:26 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:26.841191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:26.849920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:26.851437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:26.852600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63a3044f-074d-4635-9c46-3b186b6deb59 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.160613 Jul 20 14:54:26.853194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 409/813] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:27.732865 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:27.733821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:27.740011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:27.740456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:27.741209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:27.741633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:27.872700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:27.873612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:27.882098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:27.882553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:27.883348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:27.883553 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:27.883782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:27.891535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ae141a4d-5be4-468e-a2b3-7c1bf837ad99 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.162518 Jul 20 14:54:27.892031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 405/814] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:27 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:28.039754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:28.049189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:28.050671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:28.051892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f200d7bc-b0e2-40ca-aa5a-953e87e77fff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.183746 Jul 20 14:54:28.052450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 410/815] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:28.910626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:28.911704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:28.921398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:28.921836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:28.922623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:28.923068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:29.056978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:29.064844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-234ecc17-9347-43e2-bb51-eb7999fdf21e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.158490 Jul 20 14:54:29.065448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 406/816] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:28 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:29.070389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:29.070993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:29.077778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:29.078215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:29.078970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:29.079423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:29.215646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:29.232768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:29.234261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:29.235481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6bba3da9-b012-4a0d-b075-7292cca54d67 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.169164 Jul 20 14:54:29.236047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 411/817] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:30.082438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:30.083250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:30.091211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:30.091936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:30.092723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:30.093164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:30.241913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:30.250083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f1adeefb-3a1f-4e14-a538-b9eda04109d9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.171772 Jul 20 14:54:30.250707 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 407/818] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:30 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:30.254328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:30.255014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:30.261505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:30.261956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:30.262715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:30.263193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:30.401765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:30.410608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:30.412230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:30.413399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b185ce2a-94f8-4449-8b20-827c90fdcf5c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.163470 Jul 20 14:54:30.413965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 412/819] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:31.268480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:31.269024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:31.276315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:31.276767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:31.277538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:31.277970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:31.425319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:31.431805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:31.432414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:31.433201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a004275b-684a-439a-89d1-4d757486d4a8 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.168811 Jul 20 14:54:31.433817 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 408/820] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:31 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:31.439389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:31.443977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:31.443977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:31.443977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:31.572881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:31.581790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:31.583311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:31.584478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bfdcfc5b-31f0-4905-b4c2-f0ef4ce0b8b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.156533 Jul 20 14:54:31.585042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 413/821] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:32.451578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:32.452561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:32.459068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:32.459512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:32.460259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:32.460683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:32.593573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:32.601496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7aedddc5-c183-44df-b43e-566a439e41ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.154742 Jul 20 14:54:32.602054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 409/822] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:32 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:32.602771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:32.603977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:32.610713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:32.611194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:32.611962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:32.612396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:32.757321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:32.766256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:32.767787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:32.768942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fb060571-c490-43fb-b796-742976de6d44 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.170023 Jul 20 14:54:32.769541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 414/823] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:33.617895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:33.618477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:33.626463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:33.626911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:33.627713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:33.628145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:33.787487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:33.788647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:33.793758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:33.795882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:33.796328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:33.797082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:33.797505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:33.802142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5918e45c-9837-47f4-91b7-b839788ae3fb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.188090 Jul 20 14:54:33.803035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 410/824] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:33 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:33.938564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:33.947600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:33.949056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:33.950256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53113a46-95e4-40d3-b4c1-f021eb94e5c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.166588 Jul 20 14:54:33.951015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 415/825] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:34.821699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:34.822262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:34.829807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:34.830254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:34.831011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:34.831477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:34.972287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:34.972874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:34.977774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:34.979706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:34.980146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:34.981150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:34.981415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:34.985917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ebbef6d7-6aab-4a81-8109-ee720c17400b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.168190 Jul 20 14:54:34.986492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 411/826] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:34 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:35.133787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:35.142754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:35.144275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:35.145429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9fe324a1-0727-4151-b46f-87ab256f1be2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.177980 Jul 20 14:54:35.145986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 416/827] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:36.003953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:36.004711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:36.013471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:36.013471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:36.013471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:36.013826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:36.149763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:36.157606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fe89153c-1a2f-4ab3-9501-f7a2d49bbcda tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.157623 Jul 20 14:54:36.158182 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 412/828] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:35 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:36.167084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:36.167923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:36.177708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:36.178304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:36.179641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:36.180392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:36.313299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:36.321919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:36.323444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:36.324607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3738cac8-5d75-4dfd-8e47-cea3a71c1a31 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.163991 Jul 20 14:54:36.325170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 417/829] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:37.174170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:37.174752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.181806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.182292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.183096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.183541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.322868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:37.331856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b94ba8e2-f8f2-4da7-a432-93af4e36a6f0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.161633 Jul 20 14:54:37.332478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 413/830] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:37 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:37.343850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:37.344391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.350758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.351261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.352034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.352553 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.430196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:37.430705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.436919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.437363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.438134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.438546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.462444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-927a7c85-0777-479f-9c20-feeb04bca766 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/ips" status: 200 len: 149 microversion: 2.1 time: 0.137843 Jul 20 14:54:37.462988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 418/831] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 14:54:37 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/ips => generated 149 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 14:54:37.473720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a29f28ea-3635-4cf2-bc86-5a98db9dbb5e"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:54:37.474827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.480989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.481506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.482286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.482767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.487928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:37.496672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:37.498344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:37.499653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e32ce9ec-8dee-4cf1-b7f7-77e8fd1d295a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.159858 Jul 20 14:54:37.500233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 414/832] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:37.565476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: POST /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/os-volume_attachments HTTP/1.1 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 74 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:54:37.567268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {"volumeAttachment": {"volumeId": "a29f28ea-3635-4cf2-bc86-5a98db9dbb5e"}} Jul 20 14:54:37.568892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '88eb690a-bcac-4a8e-82df-86f0d0280558', 'body': {'volumeAttachment': {'volumeId': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'}}} Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:54:37.569738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:37.571074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: volume a29f28ea-3635-4cf2-bc86-5a98db9dbb5e is already attached to instances: 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24018ed9-7166-4458-99da-03fa1b10047f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.098906 Jul 20 14:54:37.572521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 419/833] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 14:54:37 2022] POST /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558/os-volume_attachments => generated 174 bytes in 100 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 20 14:54:37.578108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:37.578584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.584788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.585233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.585994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.586484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.612629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 14:54:37.704665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-403cc398-caa2-42b3-a67d-22e32f77453e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 204 len: 0 microversion: 2.1 time: 0.130083 Jul 20 14:54:37.705268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 415/834] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 14:54:37 2022] DELETE /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:54:37.714827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:37.715586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:37.722879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.723362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.724137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:37.724571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:37.866904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:37.874799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c2ea8d73-5d9d-4672-8fe4-3394f7f94612 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.163748 Jul 20 14:54:37.875404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 420/835] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:37 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:38.350800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:38.351521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:38.358443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.358894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.359697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.360132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.506034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:38.514488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-726e1e9d-7877-49bf-8d6e-c58184337c53 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.167883 Jul 20 14:54:38.515084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 416/836] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:38 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:38.518104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:38.518712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:38.525674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.526147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.526907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.527393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.658207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:38.666667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:38.668218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:38.669544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-42bad9d9-9b1e-479d-b498-934546b349a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.155395 Jul 20 14:54:38.670113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 421/837] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:38.896073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:38.897435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:38.907083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.907587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.908341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:38.908772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:38.928481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:39.053446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:39.067391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dc8f314f-195b-48af-a544-b108122b99aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.177626 Jul 20 14:54:39.067953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 417/838] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:38 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:39.532294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:39.532899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:39.539891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:39.540344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:39.541101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:39.541533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:39.678904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:39.686976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87ce318a-5907-42ad-ab1e-f27252d0dfd6 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.158579 Jul 20 14:54:39.688778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 422/839] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:39 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:39.694020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:39.695455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:39.702595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:39.703067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:39.703863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:39.704292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:39.840972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:39.849719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:39.851242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:39.852410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4718e5f5-a28b-47ea-8efd-47a160cfb9dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.162449 Jul 20 14:54:39.852967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 418/840] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:40.090078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:40.091111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:40.101328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.102079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.103442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.104120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.246468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:40.256852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74f5fbad-9536-4c00-aefe-66dbf895d96d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.173137 Jul 20 14:54:40.257754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 423/841] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:40 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:40.709767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:40.710610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:40.721625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.722364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.723688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.724463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.873894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:40.874701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:40.884003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.884703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.885509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:40.886253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:40.897894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:40.905221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9b1267d5-8de7-43c2-b054-e39d99975602 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.201984 Jul 20 14:54:40.905772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 419/842] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:40 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:41.021936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:41.031678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:41.033223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:41.034574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53c6026d-1e73-4a29-9ff9-8c44f0508e26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.167152 Jul 20 14:54:41.035178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 424/843] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:41.279513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:41.280112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:41.286334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:41.286751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:41.287577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:41.288138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:41.422153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:41.433937 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0740975f-1db4-4ee8-9867-a4fba03265b3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.158350 Jul 20 14:54:41.434481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 420/844] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:41 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:41.925886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:41.926861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:41.936383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:41.936834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:41.937617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:41.938112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:41.956626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:42.052989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:42.053590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:42.060948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:42.061304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:42.062070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:42.062519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:42.081324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:42.088981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-837449e7-5687-4643-ad77-43f10436502f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.168996 Jul 20 14:54:42.089532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 425/845] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:41 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:42.216143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:42.229320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:42.231431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:42.232746 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-46433383-8e97-48d8-b7d4-be55da21ce43 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.183681 Jul 20 14:54:42.233281 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 421/846] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:42.453643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:42.454240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:42.460390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:42.460829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:42.461574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:42.461998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:42.612244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:42.623459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad9d87b1-aa54-4b69-8ff9-b6b68da61c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.174504 Jul 20 14:54:42.624032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 426/847] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:42 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:43.106162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:43.106779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:43.113830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.114209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.114969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.115711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.251210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:43.251872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:43.258795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.259322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.260151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.260666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.272120 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:43.280202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-981573d8-896c-47cc-bdbc-c36dd376143e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.177908 Jul 20 14:54:43.280768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 422/848] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:43 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:43.407609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:43.421828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:43.424603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:43.426665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0afbae62-2327-4b18-98d1-322d135677ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.179511 Jul 20 14:54:43.427748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 427/849] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:43.640875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:43.641480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:43.648199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.648638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.649394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:43.649821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:43.782645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:43.792551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7f5029e7-fd4f-467f-be1b-24ab04a1cbc6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.155633 Jul 20 14:54:43.800330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 423/850] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:43 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:44.301296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:44.302358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:44.312247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.312482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.313299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.313739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.445934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:44.446580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:44.453337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.454044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.454992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.455670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.461873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:44.470237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b1e7bb6-5ea3-46f1-9ba1-dd57c3062eb3 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.175663 Jul 20 14:54:44.470795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 428/851] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:44 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:44.613843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:44.627741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:44.630205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:44.632044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1e0cfd5a-e5f5-48be-b737-fcb9ed6f6ab7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.189819 Jul 20 14:54:44.633007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 424/852] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:44.811684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:44.812293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:44.818576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.819025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.819807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:44.820235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:44.983220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:44.993170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6cdfdd5a-9d65-4bbb-9fb4-66ce74ac03e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.186337 Jul 20 14:54:44.993953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 429/853] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:44 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:45.487741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:45.488358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:45.495592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:45.496045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:45.496814 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:45.497250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:45.631904 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:45.639765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-248d44be-9d57-4be7-8cdd-24516ac30906 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.155944 Jul 20 14:54:45.640341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 425/854] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:45 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:45.654002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:45.654650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:45.661709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:45.662151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:45.662969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:45.663432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:45.797744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:45.806570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:45.808081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:45.809317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2287de7d-8926-45e0-81e1-49c196bbc4ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.159367 Jul 20 14:54:45.810004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 430/855] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:46.012016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:46.012541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:46.019165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.019611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:46.020374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.020809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:46.158106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:46.166729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-db68a91a-44de-4770-91c7-4c85e0e9347c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.158398 Jul 20 14:54:46.167374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 426/856] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:46 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:46.661116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:46.661962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:46.671731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.672177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:46.672943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.673373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:46.828741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:46.829487 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:46.836693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.837231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:46.838112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:46.838610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:47.113868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:47.121901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-98f88e38-9e01-4e4f-9d76-ae97c191c9fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.467343 Jul 20 14:54:47.122588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 431/857] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:46 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:47.136559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:47.145362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:47.147184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:47.148340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd00f66d-8569-4877-a828-a40702771cbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.323715 Jul 20 14:54:47.148949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 427/858] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:47.184818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:47.185498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:47.191993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:47.192440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:47.193201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:47.193629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:47.326820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:47.334757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-095d5462-de67-44cf-8bd3-87310a5f0800 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.153787 Jul 20 14:54:47.335357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 432/859] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:47 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:48.139559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:48.140841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:48.149560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.150304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.151126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.151895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.166640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:48.167635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:48.179073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.183625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.184660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.185248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.491875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:48.500190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-acffd112-6f80-4864-adf4-5275dd8f7e15 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1580 microversion: 2.1 time: 0.364605 Jul 20 14:54:48.500889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 428/860] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:48 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1580 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:48.505689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:48.506555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:48.513097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.513859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.514693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.515223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.530282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:48.540792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:48.543878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:48.545360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45439179-21ec-4fbb-834c-da4c199d8921 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.382868 Jul 20 14:54:48.547301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 433/861] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 14:54:48.553916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:54:48.555306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:48.561942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.562508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.563366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.563973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.587483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'flavor' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:54:48.732553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d83b51e0-c858-4e4c-bc94-0045369de68a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action" status: 200 len: 32 microversion: 2.1 time: 0.183575 Jul 20 14:54:48.738317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 434/862] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:54:48 2022] POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action => generated 32 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 14:54:48.752493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:48.754028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:48.763237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.764407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.765221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:48.765769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:48.772949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:48.781519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-51e98cfa-5a4d-47d8-b0ef-eab101662fca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1680 microversion: 2.1 time: 0.279254 Jul 20 14:54:48.785668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 429/863] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:48 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1680 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:49.049982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.065741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f66060f5-3087-4e6b-b951-7c2c47c58724 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.326133 Jul 20 14:54:49.066448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 435/864] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:49.248541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-9a932c62-4fcc-4daf-a343-1811a3508703 req-3e906504-4aef-4372-86b4-d49c290b4002 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88eb690a-bcac-4a8e-82df-86f0d0280558", "name": "network-vif-deleted", "tag": "288d0f95-2b49-4578-b851-0f58b382c611"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:54:49.257848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-9a932c62-4fcc-4daf-a343-1811a3508703 req-3e906504-4aef-4372-86b4-d49c290b4002 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.258536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-9a932c62-4fcc-4daf-a343-1811a3508703 req-3e906504-4aef-4372-86b4-d49c290b4002 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.271101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-9a932c62-4fcc-4daf-a343-1811a3508703 req-3e906504-4aef-4372-86b4-d49c290b4002 service nova] Creating event network-vif-deleted:288d0f95-2b49-4578-b851-0f58b382c611 for instance 88eb690a-bcac-4a8e-82df-86f0d0280558 on hv-cinder-84902 Jul 20 14:54:49.276757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-9a932c62-4fcc-4daf-a343-1811a3508703 req-3e906504-4aef-4372-86b4-d49c290b4002 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101204 Jul 20 14:54:49.277362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 430/865] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:54:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:54:49.449613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-65858971-d355-4504-b9d5-35b660e50400 req-b379232e-e063-471c-baa2-8bdd908a4945 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f3521d1-81cb-4217-b43a-b84b94911839", "name": "network-vif-deleted", "tag": "ae0ff63e-d9ee-41cd-875c-523dfce9bd0c"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:54:49.458192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-65858971-d355-4504-b9d5-35b660e50400 req-b379232e-e063-471c-baa2-8bdd908a4945 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.458613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-65858971-d355-4504-b9d5-35b660e50400 req-b379232e-e063-471c-baa2-8bdd908a4945 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.477586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-65858971-d355-4504-b9d5-35b660e50400 req-b379232e-e063-471c-baa2-8bdd908a4945 service nova] Creating event network-vif-deleted:ae0ff63e-d9ee-41cd-875c-523dfce9bd0c for instance 2f3521d1-81cb-4217-b43a-b84b94911839 on hv-cinder-84902 Jul 20 14:54:49.487756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-65858971-d355-4504-b9d5-35b660e50400 req-b379232e-e063-471c-baa2-8bdd908a4945 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105178 Jul 20 14:54:49.489763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 436/866] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:54:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:54:49.518727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:49.519903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:49.530591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.531319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.532179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.532674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.564780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:49.565974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:49.580040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.580406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.581252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.581737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.722680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.739139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-65ddef9c-7f12-4447-9fb9-a3ed64c7b0c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1512 microversion: 2.1 time: 0.225057 Jul 20 14:54:49.748826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 431/867] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:49 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1512 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:49.803723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:49.804790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:49.811927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.812765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.813839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.814485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.866750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.877044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.878487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.879377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:49.880462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-01cef6f1-fb06-42bd-ac6c-fb96c9d80b80 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.320003 Jul 20 14:54:49.882339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:49.882985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 88eb690a-bcac-4a8e-82df-86f0d0280558 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:54:49.892727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 437/868] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:49.947425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:49.956041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cdb27c7d-30f6-4605-aaf8-36a243999e6b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1332 microversion: 2.1 time: 0.155960 Jul 20 14:54:49.956635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 432/869] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:49 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1332 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:50.084355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:50.085791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:50.094430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.094808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.095736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.096176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.255851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:50.264487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-580a231f-f0a7-484a-84f1-53df238df518 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.185060 Jul 20 14:54:50.265379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 438/870] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:50.759596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:50.760894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:50.770237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.771077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.772545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.773357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.848903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:50.857985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5cfa5f4b-f6e3-4bab-b3fd-d06be542be81 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.105250 Jul 20 14:54:50.858534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 433/871] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:50 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:50.900705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:50.901333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:50.913218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.913218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.913218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.913218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.975551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:50.976193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:50.983229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.983687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:50.984466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:50.984898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.001036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:51.020573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:51.021056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.021830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 88eb690a-bcac-4a8e-82df-86f0d0280558 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:54:51.072649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.084515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a3fb9b81-94ec-46fd-9dcd-30d9f263da2c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 200 len: 1332 microversion: 2.1 time: 0.112960 Jul 20 14:54:51.085358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 434/872] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:50 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 1332 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:51.104021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.117974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.120801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.122236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab42c011-9d31-40b4-8646-dbbf2f0bb057 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1809 microversion: 2.20 time: 0.225995 Jul 20 14:54:51.122999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 439/873] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1809 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:51.282585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:51.283234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:51.289515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:51.290367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.291453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:51.292282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.456113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.473321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d4438aa-0fbd-4b89-88b4-67995568ab7d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.195417 Jul 20 14:54:51.474243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 435/874] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:51.875545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:51.876178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:51.878290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=72 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:51.886187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:51.886842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.888140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:51.888588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:51.957947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:51.962717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:51.969382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9dc46cdf-51a6-41d1-81b1-2f9ff8d5c996 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.098279 Jul 20 14:54:51.969990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 440/875] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:51 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:52.101617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:52.103094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:52.109512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.109932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.110774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.111277 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.128174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 HTTP/1.1 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:54:52.129977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 14:54:52.132211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '88eb690a-bcac-4a8e-82df-86f0d0280558'} Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:54:52.133224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 14:54:52.134504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:54:52.136595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:54:52.136595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ca31381c-df6e-4888-a1df-7aaaa9218d41 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 404 len: 111 microversion: 2.1 time: 0.034137 Jul 20 14:54:52.136595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 436/876] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:52 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 14:54:52.142573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:52.143483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:52.147842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1626e499-7477-496e-ac33-7910ae496fd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '.version_select at 0x7f7b15c8d040>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:52.150417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.151263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.152670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.152670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.182954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1626e499-7477-496e-ac33-7910ae496fd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1341317015" status: 202 len: 0 microversion: 2.1 time: 0.043710 Jul 20 14:54:52.183841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 441/877] 10.222.0.15 () {58 vars in 1220 bytes} [Wed Jul 20 14:54:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1341317015 => generated 0 bytes in 46 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:54:52.288127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:52.297248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:52.298757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:52.300219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b26e2d74-b8d2-4929-bb17-fbca75bbd4ab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1796 microversion: 2.20 time: 0.161629 Jul 20 14:54:52.301197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 437/878] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 14:54:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1796 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 14:54:52.493553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:52.494259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:52.503172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.504372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.505917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.506872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.729877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:52.738664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32a485ff-eaeb-4da4-8b5b-e50db0a0dac5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.251067 Jul 20 14:54:52.739247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 442/879] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:52.986657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:52.987327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:52.993556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.994004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:52.994781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:52.995291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:53.065053 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:53.074398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e5cc9bde-5f4b-41d8-8b0e-1d3a0c0ca4f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.091530 Jul 20 14:54:53.074992 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 438/880] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:52 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:53.755691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:53.756229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:53.763294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:53.763745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:53.764507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:53.764942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:53.898414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:53.906954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e3d5786-4462-4cb6-bfd7-602049ad2219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.155265 Jul 20 14:54:53.907600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 443/881] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:54.091727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:54.092366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:54.099839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:54.100292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:54.101073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:54.101504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:54.168476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:54.176617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f6869566-a15e-4c80-9163-953e670c4968 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088805 Jul 20 14:54:54.177163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 439/882] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:54.925281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:54.925884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:54.932622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:54.933073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:54.933834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:54.934270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:55.069386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:55.077596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-700a830a-e422-4550-a835-3da2a4c768d4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.156704 Jul 20 14:54:55.078161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 444/883] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:55.193567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:55.194134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:55.201332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:55.201788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:55.202551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:55.202982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:55.264650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:55.273032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e5e045a8-40f5-4c1b-ae96-596c3a64e461 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.083267 Jul 20 14:54:55.273530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 440/884] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:55 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:56.094375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:56.094994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:56.101535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:56.101985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:56.103078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:56.103540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:56.241638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:56.250213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-adeb711d-24a6-44ed-bdb7-5cd4b26f2204 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.159867 Jul 20 14:54:56.250756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 445/885] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:56 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:56.289941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:56.290519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:56.293133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:54:56.298705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:56.299198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:56.299963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:56.300426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:56.363157 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:56.370878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f6be792d-4c34-44a2-a4f6-f23c32fa3407 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085099 Jul 20 14:54:56.371504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 441/886] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:56 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:57.267077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:57.267685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:57.275237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:57.275687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:57.276451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:57.276862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:57.387952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:57.388532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:57.395633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:57.396388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:57.397170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:57.397602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:57.407772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:57.415977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5b2f5423-7c07-4d70-8d88-de6becd1c152 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.153007 Jul 20 14:54:57.416587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 446/887] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:57.473184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:57.481224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3e30f828-11c6-4191-8cdc-0a835abd08a7 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.097273 Jul 20 14:54:57.482102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 442/888] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:57 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:58.439866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:58.440455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:58.447359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:58.447804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:58.448581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:58.449138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:58.497862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:58.498460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:58.505366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:58.505813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:58.506579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:58.507010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:58.581104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:58.588991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dda9b1e3-72ae-4fd3-a474-cc5a1bf124d1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095054 Jul 20 14:54:58.589548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 443/889] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:58 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:54:58.609808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:58.618127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2e69194f-ecbc-4cfb-8b50-675ce904b23a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.185671 Jul 20 14:54:58.618704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 447/890] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:59.608516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:59.609142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:59.616057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:59.616506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:59.617264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:59.617695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:59.636664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:54:59.637285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:54:59.643936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:59.644371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:59.645122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:54:59.645518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:54:59.682530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:59.691175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-66e97ea3-65c5-4727-8564-0bc0dd06896e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.087908 Jul 20 14:54:59.691747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 444/891] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:59 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:54:59.795442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:54:59.803753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab22abde-84fd-4ebb-867d-2d52d79de923 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.171617 Jul 20 14:54:59.804352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 448/892] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:54:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:00.712958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:00.713953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:00.722185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:00.722604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:00.723391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:00.723816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:00.795602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:00.806689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6413665d-f020-4e8e-a1a7-f22e072c5f3c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.100317 Jul 20 14:55:00.807443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 445/893] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:00 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:00.820455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:00.821059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:00.828001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:00.828454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:00.829269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:00.829694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:00.979006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:00.988303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1062e0d0-ced1-4323-97bf-55fc182a7423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.171836 Jul 20 14:55:00.988882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 449/894] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:01.826693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:01.827407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:01.835518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:01.836049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:01.836950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:01.837459 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:01.904405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:01.912117 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9eb63eed-dc7e-4223-9d51-c135ff08c1d4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.091457 Jul 20 14:55:01.912699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 446/895] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:01 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:02.009630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:02.010644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:02.021971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:02.022716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:02.024045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:02.024788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:02.176959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:02.186809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-853eacb9-db00-469a-9053-23dfe2dbaef1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.183730 Jul 20 14:55:02.187438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 450/896] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:02 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:02.932746 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:02.934039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:02.941542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:02.942016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:02.942769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:02.943264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:03.015874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:03.024342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-97a49735-79a3-4e56-9bdf-c4ab7377f915 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.097930 Jul 20 14:55:03.024926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 447/897] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:02 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:03.206434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:03.207302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:03.217425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:03.218102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:03.219322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:03.219987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:03.368416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:03.377001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d49c4805-ffed-4fbd-ad70-22a1521c39a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.176532 Jul 20 14:55:03.377567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 451/898] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:04.044856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:04.045669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:04.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:04.053867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:04.054713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:04.055245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:04.121658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:04.133609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c8427887-266e-41be-8214-75eefe4d0077 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.094600 Jul 20 14:55:04.134440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 448/899] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:04 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:04.402118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:04.403113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:04.410255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:04.410706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:04.411526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:04.411953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:04.560417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:04.568859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-58f224c5-f197-4dfc-a5fb-6186bfc5d5c7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.173302 Jul 20 14:55:04.569381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 452/900] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:05.152367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:05.152986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:05.160109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:05.160559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:05.161321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:05.161776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:05.235884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:05.247630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-50b3f71d-bf89-4e39-9dec-3d388183020d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.099142 Jul 20 14:55:05.248195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 449/901] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:05 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:05.591977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:05.593173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:05.604495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:05.605237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:05.606626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:05.607121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:05.742643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:05.751532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c4dd8bbd-99aa-49f4-b574-ab686f8eaa86 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.167761 Jul 20 14:55:05.752121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 453/902] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:06.268102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:06.269112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:06.280027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:06.280778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:06.282034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:06.282757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:06.350888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:06.360735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-532d4afb-5d19-44ae-b22b-2bfcb99d7f45 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.099289 Jul 20 14:55:06.361302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 450/903] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:06 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:06.773526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:06.774931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:06.783895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:06.784338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:06.785344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:06.785772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:06.927375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:06.935573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b8ad4c7-a9cc-4130-bd2e-89821c1ae825 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.169760 Jul 20 14:55:06.936146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 454/904] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:07.381490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:07.382503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:07.393475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:07.394230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:07.395536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:07.396607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:07.414176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:07.461783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:07.470076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9815da28-e789-4ad6-b663-1a31e06aa445 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095138 Jul 20 14:55:07.470645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 451/905] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:07 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:07.953665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:07.954232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:07.961336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:07.961773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:07.962546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:07.962966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:08.107387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:08.115662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56fe0a74-c9c9-4059-afbe-27ba94d36cdf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1735 microversion: 2.1 time: 0.166114 Jul 20 14:55:08.116248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 455/906] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:08.487737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:08.488312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:08.498430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:08.498929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:08.500102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:08.500694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:08.579409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:08.587253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a82655bc-1d57-4649-a56c-287ae00b569c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.103488 Jul 20 14:55:08.587824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 452/907] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:08 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:09.137054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:09.137886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:09.144986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.145429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.146493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.146916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.287503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:09.296351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61ccb424-8c73-4b7a-9606-f699b5eaf24e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1715 microversion: 2.1 time: 0.165752 Jul 20 14:55:09.296903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 456/908] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1715 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:09.354511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:09.355014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:09.361405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.361849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.362616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.363064 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.608045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:09.609252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:09.619655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.620424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.621772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:09.622207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:09.685159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:09.692979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f883627-36bd-4fae-b5b4-49517470fc77 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.091393 Jul 20 14:55:09.693544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 457/909] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:09 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:10.714441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:10.715625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:10.726488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:10.727300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:10.728561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:10.729283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:10.748275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:10.808339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:10.816638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4359ac72-e1a0-411d-ab59-dda775bb192b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.108593 Jul 20 14:55:10.817204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 458/910] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:10 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:11.836776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:11.837913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:11.845778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:11.846225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:11.846977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:11.847436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:11.912296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:11.920487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-732ff5ec-5aac-4e15-bf81-3efd2fd4e3ba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.090075 Jul 20 14:55:11.921069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 459/911] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:11 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:12.941497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:12.942631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:12.952206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:12.952652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:12.953406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:12.953833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:13.014856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:13.023273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b68b4819-a01a-470e-a164-93641be02f8e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088281 Jul 20 14:55:13.023848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 460/912] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:12 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:14.049337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:14.050431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:14.060409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:14.060852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:14.061610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:14.062035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:14.126977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:14.135078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-853bda82-6b65-4a5a-8dc0-4353493152fe tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.092323 Jul 20 14:55:14.135682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 461/913] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:14 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:15.157394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:15.158122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:15.165162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:15.165607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:15.166404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:15.166805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:15.235021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:15.246654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5c08f0e8-57d5-43d0-92f1-5ed6e3b19cae tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.092916 Jul 20 14:55:15.247563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 462/914] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:15 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:16.273510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:16.274358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:16.281805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:16.282255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:16.283019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:16.283478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:16.350277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:16.359794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1ff7ece1-30ca-473f-b49e-79bc3fb3e37d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.091586 Jul 20 14:55:16.360399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 463/915] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:16 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:17.379428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:17.380304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:17.389082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:17.389536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:17.390300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:17.390735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:17.462827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:17.474608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e48c13f6-7714-48fa-89a2-0682285d433c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.100227 Jul 20 14:55:17.475267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 464/916] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:17 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:55:18.496208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:18.496981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:18.504018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:18.504468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:18.505232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:18.505663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:18.572730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:18.580592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b3bb627-5416-465f-92fc-608837f5c234 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.090804 Jul 20 14:55:18.581086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 465/917] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:18 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:19.359299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=25 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:19.421881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=13 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:19.598223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:19.598771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:19.605921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:19.606349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:19.607175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:19.607598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:19.675408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:19.683378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d906e31e-e404-49a4-ac45-fa2c795a3e25 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089071 Jul 20 14:55:19.683926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 466/918] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:19 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:20.702686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:20.703701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:20.710828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:20.711309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:20.712067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:20.712492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:20.779813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:20.787553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-90087e1b-d912-4ecc-8e4e-801068d94231 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089555 Jul 20 14:55:20.788095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 467/919] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:20 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:21.805619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:21.806434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:21.813247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:21.813688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:21.814443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:21.814865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:21.875635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:21.883232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2535c1b5-a57b-4a37-9895-d13b389f879a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.081477 Jul 20 14:55:21.883798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 468/920] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:21 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:22.902726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:22.903432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:22.905649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=59,nova_api:DELETE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:22.911975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:22.912489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:22.913367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:22.913973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:22.986192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:22.994318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f95838a0-e697-4ffd-a8e3-3f51f7f0d2dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095569 Jul 20 14:55:22.994906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 469/921] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:22 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:24.011387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:24.012057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:24.018799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:24.019286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:24.020060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:24.020782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:24.086662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:24.096163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f5c5d258-7eab-4894-b0c1-7510015a82fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088718 Jul 20 14:55:24.096866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 470/922] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:24 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:25.112803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:25.113561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:25.120266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:25.120717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:25.121479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:25.121912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:25.192060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:25.201264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0fbb4f03-9a00-43af-babb-bac91d310e50 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.092303 Jul 20 14:55:25.201850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 471/923] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:25 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:26.219620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:26.220422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:26.228767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:26.229299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:26.230285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:26.230844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:26.300504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:26.310142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-539e7c90-72da-4bf1-9e2b-c0b1de5f15cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095457 Jul 20 14:55:26.310873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 472/924] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:26 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:55:27.329767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:27.330479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:27.337149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:27.337643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:27.338503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:27.338970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:27.403795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:27.414940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-58a4b776-a405-457c-81be-c9aaaedbbece tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088951 Jul 20 14:55:27.415775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 473/925] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:27 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:28.433055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:28.433919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:28.440644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:28.441084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:28.441841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:28.442271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:28.512291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:28.521948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81a5ab2b-6076-4f59-b441-0bb0cf9dc013 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.092918 Jul 20 14:55:28.522575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 474/926] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:28 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:29.539678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:29.540323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:29.547610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:29.548059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:29.548857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:29.549266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:29.553241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:29.614003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:29.622250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-26f34c16-411c-4119-ac7c-f3063f8403dc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.086473 Jul 20 14:55:29.622832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 475/927] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:29 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:30.639773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:30.640421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:30.647508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:30.647962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:30.648730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:30.649162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:30.713454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:30.721985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24aaddd5-99d4-41d8-88cd-c51b362e51f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.086530 Jul 20 14:55:30.722557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 476/928] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:30 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:31.743121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:31.743829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:31.750925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:31.751386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:31.752148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:31.752580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:31.816330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:31.824245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-409f000f-c381-44b4-9dce-bc51e532cc3d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.084871 Jul 20 14:55:31.824818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 477/929] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:31 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:32.841768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:32.842814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:32.849486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:32.849931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:32.850699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:32.851136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:32.913549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:32.921515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c17a31a-80d0-4e7b-8b94-efa82900e9cd tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.083842 Jul 20 14:55:32.922083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 478/930] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:32 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:33.938654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:33.939384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:33.946202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:33.946645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:33.947441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:33.947866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:34.021824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:34.029828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1a0e4d2a-8399-4d1f-b0c9-25d1497b4fbb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095082 Jul 20 14:55:34.030392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 479/931] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:33 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:35.048844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:35.049744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:35.059735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:35.060281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:35.061218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:35.061684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:35.125606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:35.134506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-37c14575-16b9-4d3b-987d-342fd0e63719 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.091707 Jul 20 14:55:35.135523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 480/932] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:35 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:36.153352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:36.154143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:36.160747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:36.161189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:36.161936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:36.162363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:36.223982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:36.231732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-77119572-7d79-4d47-ae2b-ad6e8b86aca1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.082469 Jul 20 14:55:36.232299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 481/933] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:36 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:37.248714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:37.249702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:37.256444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:37.256890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:37.257668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:37.258096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:37.322067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:37.330360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f503a66-1c44-4800-84f6-1441617a2dd0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085680 Jul 20 14:55:37.331032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 482/934] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:37 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:38.349504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:38.350187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:38.357612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:38.358073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:38.358832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:38.359300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:38.429882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:38.437863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-333d1222-a9a9-4893-ad2b-c6eb7930203c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.092196 Jul 20 14:55:38.438476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 483/935] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:38 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:39.454933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:39.455751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:39.462436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:39.462889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:39.463670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:39.464107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:39.528163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:39.536207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff0de798-1cdb-4e75-9b89-7c34f5bca326 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085293 Jul 20 14:55:39.536767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 484/936] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:39 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:40.553187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:40.553909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:40.561363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:40.561815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:40.562580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:40.563015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:40.628569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:40.636666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ac2f0c7-cf7c-454c-a27c-b4428d359c4b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.087328 Jul 20 14:55:40.637268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 485/937] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:40 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:41.654179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:41.654874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:41.661611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:41.662165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:41.662874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:41.663386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:41.724436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:41.732634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59ebe258-fda2-4b92-bb54-4cd883410f16 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.082471 Jul 20 14:55:41.733209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 486/938] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:41 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:42.749018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:42.749678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:42.756359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:42.756822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:42.757579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:42.758013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:42.822699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:42.830931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b83e4de-83fb-4c62-816d-8a820c2c9637 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.086001 Jul 20 14:55:42.831528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 487/939] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:42 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:43.848085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:43.848873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:43.856972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:43.857806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:43.858941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:43.859690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:43.925912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:43.933857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f5bf2b7-846f-4758-a1c3-5d17c76936f2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089639 Jul 20 14:55:43.934443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 488/940] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:43 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:44.951018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:44.951750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:44.960245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:44.960695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:44.961461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:44.961897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:45.034396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:45.042087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7db04dc5-4910-4127-b048-65bdfd334f63 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.095149 Jul 20 14:55:45.042802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 489/941] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:44 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:46.060208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:46.060921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:46.069016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:46.069562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:46.070427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:46.070966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:46.138390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:46.146630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-97067f3e-0536-4c3b-b374-1700a2dc9fb0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.090330 Jul 20 14:55:46.147218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 490/942] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:46 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:47.164383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:47.165094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:47.172782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:47.173282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:47.174110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:47.174551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:47.240551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:47.243989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:47.249175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ca9de2a5-0d28-4d7d-a714-a3592017570d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089199 Jul 20 14:55:47.249743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 491/943] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:47 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:55:48.266177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:48.266881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:48.273732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:48.274172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:48.274928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:48.275388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:48.342666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:48.350321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1367b5de-778f-4830-b90a-cc1a552993b1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088238 Jul 20 14:55:48.351101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 492/944] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:48 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:49.367263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:49.367987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:49.374672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:49.375152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:49.375922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:49.376334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:49.440296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:49.448037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0af0454e-3f3e-4470-b2ce-7814156ac72c tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.084694 Jul 20 14:55:49.448698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 493/945] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:49 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:50.465590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:50.466228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:50.472943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:50.473385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:50.474131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:50.474557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:50.536984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:50.544797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5502108e-134c-47d0-8894-d472ecc5e785 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.083555 Jul 20 14:55:50.545353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 494/946] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:50 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:51.561767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:51.562412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:51.569155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:51.569606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:51.570369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:51.570803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:51.889065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:51.897103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3eeb774-ce15-4c92-a733-d9641bcc53c9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.339346 Jul 20 14:55:51.897673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 495/947] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:51 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:55:52.913652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:52.914982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:52.921686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:52.922133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:52.922913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:52.923350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:53.134769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:53.144782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-49897626-ddb8-481a-9364-96b05402f1f1 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.234495 Jul 20 14:55:53.145442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 496/948] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:52 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:54.161978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:54.162676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:54.164899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=56 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:54.170760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:54.171232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:54.171997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:54.172424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:54.235915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:54.243879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-04fabb97-ee6f-4f01-b03d-3d87131f7865 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085825 Jul 20 14:55:54.244423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 497/949] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:55.260130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:55.260994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:55.267511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:55.267958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:55.268710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:55.269468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:55.337051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:55.344980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bff6f044-13eb-48bc-981d-139a2d0c37ea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088706 Jul 20 14:55:55.345531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 498/950] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:55 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:56.362258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:56.362868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:56.369796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:56.370270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:56.371032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:56.371528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:56.435907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:56.443648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7b99e38c-5a70-4538-b038-49ab2c0e6df2 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085459 Jul 20 14:55:56.444208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 499/951] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:56 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:55:57.370429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:55:57.371167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:57.377586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.378034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.378794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.379257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.402369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lazy-loading 'flavor' on Instance uuid f30068fe-59db-4a1f-9ad9-040b4e98f342 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:55:57.482792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76a2cb12-d57a-43ba-9b0c-1573dea5c6d2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "POST /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342/action" status: 202 len: 0 microversion: 2.1 time: 0.116224 Jul 20 14:55:57.483998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 500/952] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:55:57 2022] POST /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342/action => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:55:57.489865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:57.490722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:57.497463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.497937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.498695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.499146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.587708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:57.595535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9aa8942-c251-4df9-bfd5-1f1765fce277 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.110198 Jul 20 14:55:57.596096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 501/953] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:57 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:57.600470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:57.601064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:57.608381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.608825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.609585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:57.610013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:57.756193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:57.764017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1c56e1d9-2291-4daf-a75e-4cef0fccd442 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.166962 Jul 20 14:55:57.764585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 502/954] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:58.612488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:58.613140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:58.620024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:58.620463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:58.621238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:58.621642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:58.686892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:58.695351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f5ff79c6-0d53-44a5-bc4b-a8aa038ec161 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.086930 Jul 20 14:55:58.695933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 503/955] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:58 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:58.782005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:58.782600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:58.789177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:58.789645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:58.790390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:58.790798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:58.934411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:58.942264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3fffd426-f806-434e-ad64-e1d6efcdf4b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.164175 Jul 20 14:55:58.942830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 504/956] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:58 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:59.555807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:59.556303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:59.562909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.563396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:59.564163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.564598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:59.593766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4f61d6e-ce0b-4e98-ab46-95f7bd0aa2a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 200 len: 194 microversion: 2.20 time: 0.041949 Jul 20 14:55:59.594322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 505/957] 10.222.0.15 () {60 vars in 1453 bytes} [Wed Jul 20 14:55:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 20 14:55:59.712013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:59.712670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:59.719706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.720161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:59.720931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.721367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:59.740025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:55:59.786498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:55:59.794743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-99432b5f-2306-4d48-a570-26e7435a1f7a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.086567 Jul 20 14:55:59.795337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 506/958] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:59 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:55:59.958920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:55:59.959551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:55:59.966177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.966627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:55:59.967415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:55:59.967850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:00.105623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:00.113698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-93835147-a813-4bda-b4d7-d76760948b53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.158689 Jul 20 14:56:00.114268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 507/959] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:55:59 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:56:00.812131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:00.812820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:00.819869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:00.820317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:00.821069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:00.821494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:00.885065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:00.893565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f4fb9c9-6119-4b5e-937c-8d1f76d780c5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.085293 Jul 20 14:56:00.894458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 508/960] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:00 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:01.131444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:01.132384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:01.138658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:01.139112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:01.139913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:01.140349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:01.291911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:01.301198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c07403e4-2c18-4035-abb9-b06545cd8741 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.173707 Jul 20 14:56:01.301790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 509/961] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:01 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:01.911143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:01.911864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:01.918909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:01.919437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:01.920247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:01.920671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:01.987555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:01.995684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9100b69e-2523-4b61-82a5-e0dbcd853c72 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088452 Jul 20 14:56:01.996268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 510/962] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:01 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:02.321544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:02.322141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:02.328348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:02.328791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:02.329580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:02.330051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:02.464087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:02.471898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3ce61b09-e354-4969-8d65-204f50fb4d5c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.154290 Jul 20 14:56:02.472469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 511/963] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:02 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:56:03.016304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:03.017121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:03.027386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:03.028115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:03.029271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:03.029915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:03.110496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:03.118990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2343adc-3f5a-4f3c-911d-80e600e041cb tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.109504 Jul 20 14:56:03.120323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 512/964] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:03 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:56:03.490369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:03.490994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:03.497643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:03.498100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:03.498859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:03.499339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:03.635691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:03.643523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-21bcaf10-230b-4932-bd63-b94ff9c0a753 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.157576 Jul 20 14:56:03.644424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 513/965] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:03 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:56:04.135785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:04.136630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:04.143392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:04.143867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:04.144954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:04.145368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:04.208766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:04.216821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d48acd9-9117-4e88-adac-04e4fe09a648 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.084906 Jul 20 14:56:04.217397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 514/966] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:04 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:04.661998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:04.662649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:04.669389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:04.669839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:04.670602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:04.671035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:04.805312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:04.813861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-022b2e25-1902-4044-852e-c6c62b8cf3c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.156009 Jul 20 14:56:04.814440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 515/967] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:04 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:05.233577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:05.234246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:05.241381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:05.241833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:05.242602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:05.243068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:05.310225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:05.317930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e30f02b5-1374-4fbe-9628-17a3fb0a94f5 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.088496 Jul 20 14:56:05.318468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 516/968] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:05 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:05.831326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:05.831937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:05.838456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:05.838916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:05.839792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:05.840224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:05.973824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:05.981953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-67608959-a5aa-44cf-a998-5a60732ac8f1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.154552 Jul 20 14:56:05.982532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 517/969] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:05 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:06.334882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:06.335638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:06.342872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:06.343326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:06.344100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:06.344533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:06.419329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:06.427425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-323abc74-6f13-4677-a364-c2743bd08a0b tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.096367 Jul 20 14:56:06.428009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 518/970] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:06 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:07.000942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:07.001550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:07.008596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:07.009053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:07.009821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:07.010251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:07.169897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:07.177648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c038933e-4ef5-49d9-b259-e8b39817a094 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.181776 Jul 20 14:56:07.178223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 519/971] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:06 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:07.444386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:07.445120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:07.452281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:07.452733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:07.453475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:07.453939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:07.521020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:07.529603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfb5aaf3-a1c2-4ece-bdf7-04fa263c51ce tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089291 Jul 20 14:56:07.530189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 520/972] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:07 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:08.195209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:08.195847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:08.202394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:08.202842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:08.203635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:08.204066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:08.336840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:08.345179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44028cb0-e174-4f9a-80db-8111cbce57ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.153864 Jul 20 14:56:08.345750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 521/973] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:08 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:08.545989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:08.546703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:08.553568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:08.554018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:08.554793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:08.555263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:08.623686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:08.631823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-111db220-b9b2-4c26-b037-8bbde42a59fc tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.089767 Jul 20 14:56:08.632372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 522/974] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:08 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 14:56:09.362042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:09.362649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:09.369085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:09.369538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:09.370366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:09.370798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:09.374753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:56:09.510609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:09.518507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-caaf531b-6291-490c-bce8-eb45aa508de1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.160452 Jul 20 14:56:09.519493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 523/975] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:09 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:09.524125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:09.524734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:09.530781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:09.531271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:09.532038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:09.532470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:19.528337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=55 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:56:19.552757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:56:24.422191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:56:24.422809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 14:56:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 14:56:28.172723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:56:28.173459 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 14:56:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 14:56:49.898880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 14:56:49.900486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 14:56:49.902150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:56:49.902966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 14:56:49.904302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 409 to user: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:56:49.905450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4225f4c2-1f2a-45e2-b218-2c149a11ddbc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 409 len: 152 microversion: 2.1 time: 100.552410 Jul 20 14:56:49.905450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 453/976] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 14:55:09 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 152 bytes in 100554 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 20 14:56:49.909049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:49.909655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:49.915965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:49.916418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:49.917180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:49.917614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:49.980021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:49.987914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-45f05ccb-2c47-4457-ae90-0d3e159bbc8f tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.083416 Jul 20 14:56:49.988515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 454/977] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:49 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:49.997981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:49.998619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:50.004717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.005266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.006025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.006474 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.083972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 14:56:50.086285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 14:56:50.087951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 14:56:50.088894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued Jul 20 14:56:50.090365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 14:56:50.091803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 409 to user: Cannot 'detach_volume' instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 while it is in vm_state rescued {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 14:56:50.092769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c66ef083-67a0-4305-910b-412e1278c93c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 409 len: 152 microversion: 2.1 time: 40.572294 Jul 20 14:56:50.096908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 524/978] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 14:56:09 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 152 bytes in 40573 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 20 14:56:50.104509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-4c7b9da4-524d-4a70-b464-b5b3b73c1a49 req-9ce6f1cf-0bcc-42c1-a2e6-e9bd7cc49303 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "name": "network-vif-unplugged", "status": "completed", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:56:50.114370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4c7b9da4-524d-4a70-b464-b5b3b73c1a49 req-9ce6f1cf-0bcc-42c1-a2e6-e9bd7cc49303 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.114889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4c7b9da4-524d-4a70-b464-b5b3b73c1a49 req-9ce6f1cf-0bcc-42c1-a2e6-e9bd7cc49303 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.126856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-4c7b9da4-524d-4a70-b464-b5b3b73c1a49 req-9ce6f1cf-0bcc-42c1-a2e6-e9bd7cc49303 service nova] Creating event network-vif-unplugged:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 14:56:50.132663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-4c7b9da4-524d-4a70-b464-b5b3b73c1a49 req-9ce6f1cf-0bcc-42c1-a2e6-e9bd7cc49303 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038160 Jul 20 14:56:50.133675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 525/979] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 14:56:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 14:56:50.139406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:50.139987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:50.144422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:56:50.146662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.147150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.147918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.148361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.152432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-169cf06a-16b4-4103-9cc3-e717a3a40714 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.158495 Jul 20 14:56:50.153079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 455/980] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:56:49 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:56:50.158040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:56:50.158537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:56:50.164559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.165032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.165845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:56:50.166301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:56:50.799796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:56:50.840480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 14:57:00.144813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:57:00.164304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=6 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:57:00.804266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=12 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:57:00.878735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=22 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:58:03.667412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:58:03.668044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 14:58:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 14:58:12.951342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:58:12.952454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 14:58:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 14:58:51.004535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c171335-10bb-4da8-af4e-faf46a6b83a6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 202 len: 0 microversion: 2.1 time: 120.869163 Jul 20 14:58:51.005375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 526/981] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 14:56:50 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 120871 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:58:51.098986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 14:58:51.099638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:58:51.106169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.106622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.107450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.107999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.166868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-abee9857-f33e-449b-826a-210803a0490e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 202 len: 0 microversion: 2.20 time: 121.012400 Jul 20 14:58:51.167570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 456/982] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 14:56:50 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 121014 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 20 14:58:51.173649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:58:51.174627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:58:51.182578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.183454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.183589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d89b2255-9cc6-4297-980d-2629e1f51870 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action" status: 202 len: 0 microversion: 2.1 time: 0.175761 Jul 20 14:58:51.184378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 527/983] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 14:58:51 2022] POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 14:58:51.184975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.185559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.193094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:58:51.193957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:58:51.202769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.203852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.204670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.205099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.257196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:58:51.269305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-769bfbf5-bd48-419d-a8fd-ad0c68441763 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.099905 Jul 20 14:58:51.270619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 457/984] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:58:51 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:58:51.276864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:58:51.277633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:58:51.284858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.285287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.286230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.286673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.364307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 14:58:51.372249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eccc9fbd-3587-4267-96aa-65ce5fbb8085 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.184913 Jul 20 14:58:51.372819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 528/985] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 14:58:51 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1569 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 14:58:51.377953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 14:58:51.378435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 14:58:51.384439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.385082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:58:51.385841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 14:58:51.386271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 14:59:01.285002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:59:01.306647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=11,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:59:01.381696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:59:01.405393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 14:59:44.432181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:59:44.433114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 14:59:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 14:59:48.183173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 14:59:48.183953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 14:59:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:00:51.676163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:00:51.719909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:01:01.682336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:01:01.724375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:01:23.678230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:01:23.678968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:01:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:01:32.964052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:01:32.964745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:01:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:01:40.861769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:01:40.862421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:01:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:01:48.720385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:01:48.721216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:01:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:02:51.935061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:02:51.937419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:02:51.938837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:02:51.939754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:02:51.940870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.942047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:02:51.943094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) Jul 20 15:02:51.944276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-03814477-faa6-4bdb-91a6-288cb1bad116) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 240.666846 Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:02:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:02:51 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-aa7dc66c-26ee-44bd-919e-a8d592894d15 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:02:51.945635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:02:51.946314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 458/986] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 14:58:51 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 240669 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:02:51.946314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:02:51.946314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:02:51.954114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:51.954566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:51.955532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:51.955996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:51.973853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:02:51.976419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:02:51.977938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:02:51.978665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:02:51.980178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.981293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:02:51.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) Jul 20 15:02:51.988103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16bb2c37-6991-400c-af15-4f50f6a7f073) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 240.604104 Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:51 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-8512bbf6-bbed-4593-8df3-a173b798c31f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:02:51.990704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:02:51.991619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 529/987] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 14:58:51 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 240606 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:02:51.991619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:02:51.992681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:02:52.000701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.000701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.001952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.002511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.079659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:02:52.090177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.108535 Jul 20 15:02:52.091205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:02:52.091205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:02:52.092192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-eafc4d08-bfc6-47ee-a796-485260ba8b90 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:02:52.092192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:02:52.092192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:02:52.092511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 530/988] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:02:51 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:02:52.097426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:02:52.098103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:02:52.104647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.105151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.105922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.106349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.217139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.275995 Jul 20 15:02:52.217677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:02:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:02:52.217677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:02:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:02:52.218270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-5dcf4d53-8fd4-4c9a-9189-9aaad2c43d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:02:52.218270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:02:52.218270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:02:52.218554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 459/989] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:02:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:02:52.223224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:02:52.223716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:02:52.230205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.230652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.231448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.231879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.279780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:02:52.287819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.194205 Jul 20 15:02:52.288339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:02:52.288339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:02:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:02:52.288986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-9da49a3e-a581-441a-af1d-c8320b292215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:02:52.288986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:02:52.288986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:02:52.289287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 531/990] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:02:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:02:52.293924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:02:52.294412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:02:52.300581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.301024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:02:52.301785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:02:52.302249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:03:02.227584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:03:02.251765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=8 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:03:02.298549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:03:02.321616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=16 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:03:04.444448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:03:04.445172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:03:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:03:08.192148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:03:08.192832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:03:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:04:43.687929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:04:43.688835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:04:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:04:52.282181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:04:52.485002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:04:52.974722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:04:52.974722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:04:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:05:00.870240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:05:00.870924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:05:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:05:02.287392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:05:02.489648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:05:08.733282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:05:08.733766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:05:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:06:24.453001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:06:24.453809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:06:28.202735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:06:28.203450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:06:52.731344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:06:52.733562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:06:52.735023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:06:52.735978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:06:52.737381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.738795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:06:52.740091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) Jul 20 15:06:52.741558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f6d11b-c603-470e-b68f-1f3b4bef0725) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 240.515687 Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-dee81f01-4bd6-4479-a988-e2407558c3a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:06:52.743386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:06:52.744212 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 460/991] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:02:52 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 240517 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:06:52.766528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:06:52.768749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:06:52.771685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:06:52.775893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:06:52.778543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.780944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:06:52.783537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) Jul 20 15:06:52.787915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-48f04351-4ab0-4870-9b44-84b5ebeedad5) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 240.480189 Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-9f316047-8b5e-4234-826b-837e1b19a449 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:06:52.790103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:06:52.791771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 532/992] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:02:52 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 240482 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:06:52.791771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:06:52.791771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:06:52.791771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.792533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:52.792533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.792533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:52.865502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd610b017e7d4bd786db6bcd08e60722', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c8531e2e07a47eaae4c66d0b82df2d9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:06:52.875212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 200 len: 1226 microversion: 2.1 time: 0.102326 Jul 20 15:06:52.875740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:06:52.875740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:06:52.876320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-c5279ef3-4f4c-4eb5-b934-3e68f0fb74f4 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:06:52.876320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:06:52.876320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:06:52.876745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 533/993] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:06:52 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:06:52.881610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:06:52.883010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:06:52.890462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.890909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:52.891699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.892167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:52.893656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:06:52.894199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:06:52.900500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.900951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:52.901714 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:52.902146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:53.008373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.270874 Jul 20 15:06:53.008909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:06:53.008909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:06:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:06:53.009485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-dad155e0-b32a-4653-9aef-4aaa0c008e63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:06:53.009485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:06:53.009485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:06:53.009775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 461/994] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:06:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:06:53.014652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:06:53.015152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:06:53.021371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:53.021811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:53.022577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:53.023014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:53.057845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:06:53.070044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.192162 Jul 20 15:06:53.070702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:06:53.070702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:06:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:06:53.071614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-a381d284-f4fa-40fb-9ca2-b2bd76f8e6ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:06:53.071614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:06:53.071614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:06:53.072076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 534/995] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:06:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:06:53.078859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:06:53.079620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:06:53.086607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:53.087082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:06:53.087857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:06:53.088283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:07:03.018911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:07:03.041652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=8 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:07:03.084475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:07:03.106625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=16 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:08:03.697380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:08:03.698453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:08:12.983549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:08:12.984081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:08:20.884466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:08:20.885482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:08:28.742832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:08:28.743868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:08:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:08:51.257774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:08:51.299831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:08:52.993480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:08:52.995651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:08:52.997136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:08:52.998037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:08:52.999269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.000499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.001532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) Jul 20 15:08:53.002699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-337f20d5-e12d-4c52-90d6-d0ba9e8c8064) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 119.986497 Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:08:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:08:52 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-04cf3e68-4817-4a08-9fe7-e58c986493f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:08:53.003829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:08:53.004697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 462/996] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:06:53 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 119988 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:08:53.004697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:08:53.004697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:08:53.010484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.010922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.011709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.012137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.034053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:08:53.036311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:08:53.037733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:08:53.038616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:08:53.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.041366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.042392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) Jul 20 15:08:53.043618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c43fc6d4-226d-43d2-9113-ba10747ae923) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 119.965126 Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-1403608c-e7bb-4cd6-9f31-2399f4018710 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:08:53.044682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:08:53.045215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 535/997] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:06:53 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 119967 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:08:53.046908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:08:53.047559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:08:53.054654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.054997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.056152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.056353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.070492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:08:53.072242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:08:53.073752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:08:53.074679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:08:53.075938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.033573 Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-5fdf67a0-a67b-42a2-9412-fe66380544fa tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:08:53.077052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 536/998] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:08:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:08:53.079205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:08:53.079812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:08:53.095132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.095132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.095132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.095132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090933 Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:08:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:08:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-d76f3ca6-f408-4fb7-b47b-e36eb58de0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:08:53.095491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 463/999] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:08:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:08:53.096213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:08:53.096698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:08:53.103443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.103882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.104641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.105063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.237405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:08:53.245365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1569 microversion: 2.1 time: 0.169664 Jul 20 15:08:53.245898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:08:53.245898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:08:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:08:53.246479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-3825a3a3-b952-45ba-b656-f2002276ce50 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:08:53.246479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:08:53.246479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:08:53.246794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 537/1000] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:08:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:08:53.251159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:08:53.251647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:08:53.257787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.258238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:08:53.259005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:08:53.259775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:09:03.101368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:09:03.123897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=12 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:09:03.254734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=6 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:09:03.279216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=16 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:09:44.463700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:09:44.464679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:09:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:09:48.213089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:09:48.214161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:09:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:51.461482 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:51.623196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:53.168186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.170447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.172197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.173048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.174258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.175946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.177133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) Jul 20 15:10:53.178788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-fd9595e3-f9af-49ec-abd0-89fbacc8df35) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 120.079884 Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-e0c4ed9b-71b5-4903-9833-a69075831003 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.183486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.184396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 464/1001] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:08:53 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 120082 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.184396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.184396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.190361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.190875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.191683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.192120 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.209627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:53.211823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.213716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.214354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.215972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.217755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.220285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) Jul 20 15:10:53.222488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3bc2604b-0ff8-40d3-963a-615dbd4ba706) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 119.965735 Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-f9a0a720-e6cb-4420-bd1d-d0ea7284e53a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.223848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.224775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 538/1002] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:08:53 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 119967 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.224775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.224775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.242410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.242874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.243655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.244086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.259851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:53.261706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.263904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.265191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.048095 Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-ec0a7460-34c8-4885-bb28-4b6248fa4a68 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.266409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 539/1003] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.298131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.123528 Jul 20 15:10:53.298733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:53.298840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:53.299453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-bb38cef9-517a-4e18-af6c-e87846e1cffc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:53.299453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.299453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.299704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 465/1004] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.304420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.304907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.310998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.311484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.312268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.312697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.394996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:53.396048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.396683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.402814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.403292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.404040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.404486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.452135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.454372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.456196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.457021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.458755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.461370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.463640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) Jul 20 15:10:53.465418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1febffa8-d43b-4cf3-91c8-3194681781f4) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.155476 Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-0084ed62-630a-417d-9cb7-03f8810b0241 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.466902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.467601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 466/1005] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:53 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 157 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.467601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.467601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.468492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.468989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.469752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.470184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.538890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:53.546360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.280859 Jul 20 15:10:53.546977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:53.546977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:53.547573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-a6e5908f-14f3-4267-9494-f484c5cdc215 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:53.547573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.547573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.547835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 540/1006] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.552220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.552866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.553994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:53.558857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.559334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.560173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.560603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.611953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.615499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.617603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.618654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.620662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.622245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) Jul 20 15:10:53.623878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7ba202e-5cdd-4523-88bd-7dd7ab7f4679) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.158794 Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-f9a1fb82-152a-4609-b176-bcce2140b27c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.625458 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.626093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 467/1007] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:53 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 160 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.626093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.626093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.629839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.630293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.631095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.631537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.645855 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.647318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098562 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.647520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.649239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.650185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.651901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-8fb5fcc2-7ff9-44a4-9c8a-be3473e0a6b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.654066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 541/1008] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.654832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.654832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.033938 Jul 20 15:10:53.655134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.655308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:53.655308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:53.655308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-ce3736ce-dff5-4cc4-9775-0d35b889c324 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:53.655308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.655308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.656014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.656229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 468/1009] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.659083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.659573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.660051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.660492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.661242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.661684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.665535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.665981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.666743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.667388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.745837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:53.799034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:53.803039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.805230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:53.806548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.807208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.808284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.809491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.810930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) Jul 20 15:10:53.812556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.813869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c2f007e0-dd2e-4695-8ef4-3f41861eaa49) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.813869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.151837 Jul 20 15:10:53.813869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:53.813869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:53.814213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.157787 Jul 20 15:10:53.814335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-aecf88dc-dfa8-4d0a-a35b-1dea7519e6b8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:53.814335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.814335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.814335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 469/1010] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:53 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 153 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-31b240c5-6442-495f-86d7-5b87b3c056c8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.814663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 542/1011] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.815718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.815865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.816012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.816119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.819892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.820223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.820343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.820659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.821158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.821390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.821594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.822004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.903330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:53.907727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.098099 Jul 20 15:10:53.908281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:53.908281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:53.908841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-4be04d0e-8851-4cb5-9709-000f00a4e286 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:53.908841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.908841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.909099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 543/1012] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.913635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.914269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.920550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.920988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.921870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.922306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.936127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:53.937761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.939166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.939923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:53.941157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.029499 Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-d4df1a04-cd24-48ff-9ba5-177534f17bea tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:53.942229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 544/1013] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.944879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.945487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.951169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.951620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.952376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.952836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.958141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:53.960346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:53.961725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.962342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:53.963440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.964607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.965957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) Jul 20 15:10:53.967481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c0f402e1-bdd8-4b06-b7c9-26bf44717b84) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.152994 Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:53 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-027c69b4-ce54-4c77-bb96-cd947d789b81 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:53.968985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:53.969615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 470/1014] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:53 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 154 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:53.969615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:53.969615 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:53.974161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.974609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:53.975411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:53.975847 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.055923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.087649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:54.095232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.153774 Jul 20 15:10:54.095738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:54.095738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:54.096312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-b140940c-4be8-4071-bf9f-fe94a260c2c6 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:54.096312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.096312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.096580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 545/1015] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.100980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.101474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.107131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.107588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.108352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.108785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.113007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:54.115187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.116576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.117211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.118240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.119740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.121059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) Jul 20 15:10:54.122415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-683a2165-8a36-4819-8342-6b929c6bdc65) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.152777 Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-597c21b9-4ff2-443a-a321-b2fe5a5c47a1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.124401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.125355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 471/1016] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:53 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 154 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.125355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.125355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.130344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.130733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.131524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.131963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.189235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.213316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094567 Jul 20 15:10:54.213869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:54.213869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:54.214480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-08f10867-f144-482c-b94a-cdeb74ea4b4f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:54.214480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.214480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.214734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 472/1017] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.219038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.219686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.225662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.226114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.226873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.227347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.240854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:54.242520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.243997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.244952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.246715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.028513 Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-c41a7e4c-ca40-487f-8847-dd7819e550f9 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.248568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 473/1018] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:54.249787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.251951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.252711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.254637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.257367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.259608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) Jul 20 15:10:54.261221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9b87f4f-4bcd-4706-ba1b-430bd483f47b) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.153319 Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-a54570fd-6004-4491-af8c-cbe5bb302371 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.262598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.263237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.263352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 546/1019] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 155 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.263446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.263585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.263585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.263872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.263872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.263872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.263872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.265128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.265617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.266379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.266817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.347001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.391513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:54.399225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.151168 Jul 20 15:10:54.399739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:54.399739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:54.400324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-81c9a01e-7da8-4936-ba91-d6c553b4af99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:54.400324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.400324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.400594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 474/1020] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.404350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.405226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.405721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:54.406938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.408375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.408998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.410056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.411341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.412929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) Jul 20 15:10:54.414702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-efb50cff-d7a7-4558-ad0b-a15b059bdfda) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.155836 Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-54c2aa0a-0cac-44ae-85b3-3473fd932d16 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.416221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.416798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 547/1021] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 157 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.416902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.416902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.416902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.416902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.417393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.417393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.422517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.422963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.423765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.424194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.498209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.501516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090792 Jul 20 15:10:54.502050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:54.502050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:54.502657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-16bc5d78-1d18-424d-851e-db4aaf23e0b1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:54.502657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.502657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.503000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 548/1022] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.507644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.508241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.514499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.514942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.515700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.516150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.529473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:54.531076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.532622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.533370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.534633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.028773 Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-a503163e-4b7b-4176-9f5e-e383d059a60d tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.535754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 549/1023] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.537880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.538484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.544164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.544604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.545354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.545783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.556099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:54.558260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.559931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.560552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.561665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.563920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.565419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) Jul 20 15:10:54.567003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ddcc2261-45fd-4a1d-9608-e10d6512a196) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.158373 Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-d2d82488-3a6a-4e89-b557-bc38b318c8d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.568343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.568905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 475/1024] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 160 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.568905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.568905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.572234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.572680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.573441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.573876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.653914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.681463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:54.689222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.154740 Jul 20 15:10:54.689781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:54.689781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:54.690396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-a0839176-e6e7-4018-a578-9f40b06dce41 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:54.690396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.690396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.690670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 550/1025] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.695088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.695547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.701418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.701854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.702602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.703189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.712193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:54.714429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:54.716048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.716706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.717908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.719068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.720259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) Jul 20 15:10:54.721868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b19810d1-a3f5-4399-9845-cd6b31ad93c1) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.155229 Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (ip 10.222.0.15) !!! Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e (10.222.0.15) Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-83bc87c6-f044-4a62-a515-8c088b9fcfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Unhandled error: OSError: write error Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.723427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.724121 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 476/1026] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 0 bytes in 157 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.724121 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.724351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.730583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.731035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.731838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.733727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.796407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.811791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092258 Jul 20 15:10:54.812322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (ip 10.222.0.15) !!! Jul 20 15:10:54.812322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface (10.222.0.15) Jul 20 15:10:54.812941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-2724d385-6d32-4cfa-b692-cb2213ec0d2d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Unhandled error: OSError: write error Jul 20 15:10:54.812941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.812941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.813206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 477/1027] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 0 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.817677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.818275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.824721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.825198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.825960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.826388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.839743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:54.841240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.842637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.843578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:54.844815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.028653 Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (ip 10.222.0.15) !!! Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 (10.222.0.15) Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-6cd2c61a-1762-42a6-9c4c-2efd1f9f1575 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Unhandled error: OSError: write error Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.846202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 478/1028] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.848007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.848595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.854209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.854645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.855420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.855843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.856326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:54.858577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:54.860219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.860836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:54.861888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.863151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) Jul 20 15:10:54.865819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f97c9b69-d0df-4843-84a9-f682ac07a022) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.169133 Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (ip 10.222.0.15) !!! Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 (10.222.0.15) Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: CRITICAL nova [None req-f1783771-e6a2-4d7a-94c2-457301747073 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Unhandled error: OSError: write error Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova OSError: write error Jul 20 15:10:54.867703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova Jul 20 15:10:54.868212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 551/1029] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 0 bytes in 171 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:54.868212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:54.869016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:54.875807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.876255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.877018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:54.877484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:54.960143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:54.989705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:54.997340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.152742 Jul 20 15:10:54.997849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (ip 10.222.0.15) !!! Jul 20 15:10:54.997849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:10:54 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 (10.222.0.15) Jul 20 15:10:54.998423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: CRITICAL nova [None req-048e1223-1a1d-401f-8bab-1a26635c2a28 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Unhandled error: OSError: write error Jul 20 15:10:54.998423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova OSError: write error Jul 20 15:10:54.998423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova Jul 20 15:10:54.998749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 479/1030] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:54 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Jul 20 15:10:55.003189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.003813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.009582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.010031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.010794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.011286 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.016253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e HTTP/1.1 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:55.018429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:10:55.019882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '6b566c6c-a418-4384-b99e-c7295e754e92', 'id': 'a29f28ea-3635-4cf2-bc86-5a98db9dbb5e'} Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.020525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.021551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.022676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.023750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) Jul 20 15:10:55.024818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.026149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4ddf2cb7-5180-4928-aa79-79aeddd99129) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.027226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9a95d3ff-8b02-4245-b457-9b7fd263a823 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e" status: 400 len: 269 microversion: 2.1 time: 0.164432 Jul 20 15:10:55.027790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 552/1031] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments/a29f28ea-3635-4cf2-bc86-5a98db9dbb5e => generated 269 bytes in 166 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:10:55.033249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.033742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.040104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.040580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.041403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.041868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.090077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'services' on Instance uuid 3dfe0713-c262-4daa-8e0d-d439f6e68f7d {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:55.126238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7d16aa19-2720-41db-b286-fa654da43bb9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.097356 Jul 20 15:10:55.126817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 553/1032] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.131553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.132468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.138564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.139020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.139804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.140236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.145061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 HTTP/1.1 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:55.147261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:10:55.148956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d', 'id': '44b5537a-5392-4a04-990f-113d3e14c717'} Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.149721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.151005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.152530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.153872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) Jul 20 15:10:55.156156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.157369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-962539e6-d8de-4cbd-b480-3fa16e0e12a0) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.157369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-62fff3af-3f69-4796-a914-1d85677f9b90 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717" status: 400 len: 269 microversion: 2.20 time: 0.149236 Jul 20 15:10:55.157369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 480/1033] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:10:54 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments/44b5537a-5392-4a04-990f-113d3e14c717 => generated 269 bytes in 150 msecs (HTTP/1.1 400) 9 headers in 383 bytes (1 switches on core 0) Jul 20 15:10:55.157632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.157736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.157736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:55.157905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.159666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.160648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.163076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.164534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.164534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2556963e-d81a-4353-9e54-c07ae0e05cf0 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.028725 Jul 20 15:10:55.164534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 554/1034] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:55.164791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.164791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.164952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.165034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.165034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.165184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.166732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:10:55.168393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.168849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.169714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.170149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.199835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c4d90b9a-5070-4828-9bb6-7ab61190952b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041823 Jul 20 15:10:55.200393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 555/1035] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.204985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.205893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.211740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.212183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.212939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.213403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.242559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4bd409f2-6f0f-43a6-abfc-064c2ff72985 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041110 Jul 20 15:10:55.243141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 556/1036] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:55.247564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.248054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.253805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.254381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.254993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.255464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.272081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:10:55.308648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:55.316417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a4c317c6-f674-4aa7-ba42-9e4157438ea7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1563 microversion: 2.1 time: 0.166271 Jul 20 15:10:55.316980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 481/1037] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1563 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:10:55.343953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e8b3fbc-0d85-44d1-925d-3e11357b5f48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099748 Jul 20 15:10:55.344722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 557/1038] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.349296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.349790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.355909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.356381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.357143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.357578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.366841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.367380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.371622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:55.373170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.374928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.376104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.377965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.380430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.380531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.380629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.380731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b4c15bc-3e72-4434-b80e-0d83cf8d1b20 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.028929 Jul 20 15:10:55.380832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.380832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.381000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 558/1039] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:55.381000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.381000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.387004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.387533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.388296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.388731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.414357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:10:55.459086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lazy-loading 'services' on Instance uuid e827c810-220b-4fe1-9d16-01bc8372b933 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:55.495559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4fbc806f-eb16-4f0d-9c20-e82272d23935 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 204 len: 0 microversion: 2.1 time: 0.118960 Jul 20 15:10:55.496597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 559/1040] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:10:55.535974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6b11bfcb-768e-4ec5-9306-f394aa6e407e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6" status: 202 len: 0 microversion: 2.1 time: 0.217786 Jul 20 15:10:55.536683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 482/1041] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 => generated 0 bytes in 219 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:10:55.541804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.542936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.549376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.549884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.550710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.551245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.562255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-fea63411-f234-4275-9429-b4ddc422c95c req-088313a3-4059-4c09-ae59-57b031fd4af0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:10:55.570354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-fea63411-f234-4275-9429-b4ddc422c95c req-088313a3-4059-4c09-ae59-57b031fd4af0 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.570831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-fea63411-f234-4275-9429-b4ddc422c95c req-088313a3-4059-4c09-ae59-57b031fd4af0 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.580493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-256f294c-abba-4e70-ab0c-2b27c41694dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042534 Jul 20 15:10:55.581039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 483/1042] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.582465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-fea63411-f234-4275-9429-b4ddc422c95c req-088313a3-4059-4c09-ae59-57b031fd4af0 service nova] Creating event network-changed:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 15:10:55.585534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.586434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.587532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-fea63411-f234-4275-9429-b4ddc422c95c req-088313a3-4059-4c09-ae59-57b031fd4af0 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089128 Jul 20 15:10:55.588097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 560/1043] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:10:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.592357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.592805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.593489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.593610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.594021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.594129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.600769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.601251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.602049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.602481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.622547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-21a74146-c0d8-4bad-91da-4400ccd742ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040504 Jul 20 15:10:55.623132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 484/1044] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:55.627542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.628034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.634005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.634445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.635233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.635665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.648947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:55.650569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.652170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.652912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.654119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.655370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.655370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a550c243-7282-4d72-9e42-f5de03311873 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.027926 Jul 20 15:10:55.655370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 485/1045] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:55.656909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.657446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.663151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.663540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.664301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.664726 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.687536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lazy-loading 'services' on Instance uuid e827c810-220b-4fe1-9d16-01bc8372b933 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:55.690535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:10:55.738434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.compute.api [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Instance is already in deleting state, ignoring this request Jul 20 15:10:55.739204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5f7a3630-7ee5-4339-a3d0-10131a9431fe tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 204 len: 0 microversion: 2.1 time: 0.085819 Jul 20 15:10:55.739760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 486/1046] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:10:55.743677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.744604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.745092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Request method failure captured: Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 HTTP/1.1 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2Ba2Da-ehBHDesIrcgDFA3lA1KgQrWEUFDIeCHqgOF9KhPcNdR6P2u1G8xQyRZGgAgjeXMgX_c5FhDetQdAHZ8Bt6WCqSXoS2rFn_mggGc7thSZvMlISdU0eg7gYUW8SiIF4C3j_SUhctUlKAg__5eNzle3kG2QmJW6mEl_ZctiuMq0 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:55.746052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}] Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: bc2f1a0d431341d6a5d1a8c5de131ece Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:55.747801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': 'e827c810-220b-4fe1-9d16-01bc8372b933', 'id': 'eaa89bcc-652b-4a03-941b-8440920143e6'} Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.748469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:55.749625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.750870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.753207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) Jul 20 15:10:55.754663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.756163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1ff405eb-34c5-4e45-890a-1a38d224eeb9) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.756163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76786703-a28f-4d77-aefd-8fa0fb196ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6" status: 400 len: 269 microversion: 2.1 time: 0.158155 Jul 20 15:10:55.756163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 561/1047] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 => generated 269 bytes in 159 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:10:55.756449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.756449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.756449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.756449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.756820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.756820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.764067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.764544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.765419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.765884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.801275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-705d92da-ead0-4aa0-9171-b0cb8ed44b22 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052108 Jul 20 15:10:55.802065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 562/1048] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.806652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.807586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.814777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.815550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.816509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.817263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.845618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:55.859780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f91e5d4f-63a0-4599-8e4d-3a279bea1fb5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056729 Jul 20 15:10:55.860352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 563/1049] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:55.865170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.866660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.872837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.873285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.874045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.874475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.904894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1d995f8-3d12-45cf-a27e-cb059037f2c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043300 Jul 20 15:10:55.905693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 564/1050] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:55.911083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.911568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.919170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.919637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.920448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.920879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.927218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6b4d70fc-659a-4f0b-8cd8-1a4cfa9d1f7c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 202 len: 0 microversion: 2.1 time: 0.185802 Jul 20 15:10:55.931180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 487/1051] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:10:55.937547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.938199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.945648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.945747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.946489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.947252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.961773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:55.963602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.965278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:55.966119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:55.967367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:55.969748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:55.971484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9004e69e-918a-4d6d-ab78-60627035010a tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.038228 Jul 20 15:10:55.972764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 488/1052] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:55.978625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:55.979273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:55.986445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.986891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:55.987684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:55.988108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.015531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:10:56.043478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lazy-loading 'services' on Instance uuid e827c810-220b-4fe1-9d16-01bc8372b933 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:56.067402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.compute.api [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Instance is already in deleting state, ignoring this request Jul 20 15:10:56.068346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-02483b45-dfa2-497e-9b2b-5ec9a92e12c3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 204 len: 0 microversion: 2.1 time: 0.094475 Jul 20 15:10:56.069021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 489/1053] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:10:56.073998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.074751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.081918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.082816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.083827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.084272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.154084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Request method failure captured: Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 HTTP/1.1 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2Ba2Da-ehBHDesIrcgDFA3lA1KgQrWEUFDIeCHqgOF9KhPcNdR6P2u1G8xQyRZGgAgjeXMgX_c5FhDetQdAHZ8Bt6WCqSXoS2rFn_mggGc7thSZvMlISdU0eg7gYUW8SiIF4C3j_SUhctUlKAg__5eNzle3kG2QmJW6mEl_ZctiuMq0 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:56.156518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}] Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: bc2f1a0d431341d6a5d1a8c5de131ece Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:56.158008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': 'e827c810-220b-4fe1-9d16-01bc8372b933', 'id': 'eaa89bcc-652b-4a03-941b-8440920143e6'} Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.158889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.159966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.162794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.164840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) Jul 20 15:10:56.167519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.169680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-daf88a8d-5bbf-4685-91de-bd982600179b) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:56.170011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59d61ab8-0098-40f7-aa59-fa6c9406c98f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6" status: 400 len: 269 microversion: 2.1 time: 0.263035 Jul 20 15:10:56.170813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 565/1054] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:55 2022] DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 => generated 269 bytes in 264 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:10:56.176322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.177230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.184963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.185420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.186323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.186750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.205010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:56.224083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-afc643ba-efb6-442a-ac3a-0e9ccae0713d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052254 Jul 20 15:10:56.225180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 566/1055] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:56.231290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.232411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.241741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.242499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.244040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.244660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.270126 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:56.272334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:10:56.273745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.274388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.275492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.276686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.277761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) Jul 20 15:10:56.278838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.282965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-6b5a5d2f-c8e7-4871-8d08-d238e6936a72) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:56.282965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b565675a-ab8e-4995-b442-fddf0f5b69cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 0.203880 Jul 20 15:10:56.282965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 490/1056] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:56 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 205 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:10:56.282965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.283592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-43a858d8-9751-44bb-97ab-392bd2e927c4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052918 Jul 20 15:10:56.284319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.284699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 567/1057] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:56.288697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.288697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.289103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.289293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.289460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.289882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.296052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.296482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.297553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.298174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.324311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b6460f1a-dc38-4294-b61f-67e8cce40542 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048467 Jul 20 15:10:56.324920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 491/1058] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:56.332905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.333373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.339815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.340268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.341032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.341488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.355917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:10:56.357539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.358949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:56.359886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.361186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.362514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:56.362514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-840bb1cb-de66-4cdc-bbfb-63a8ada6e937 tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.032866 Jul 20 15:10:56.362514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 492/1059] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:56 2022] DELETE /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:56.364686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.365174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.371753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.372191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.373071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.373501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.402790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:10:56.404065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lazy-loading 'services' on Instance uuid e827c810-220b-4fe1-9d16-01bc8372b933 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:56.472170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.compute.api [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Instance is already in deleting state, ignoring this request Jul 20 15:10:56.473271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bc739d58-2378-48a5-ad35-8e1b051d5186 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 204 len: 0 microversion: 2.1 time: 0.112637 Jul 20 15:10:56.474446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 493/1060] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:10:56 2022] DELETE /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:10:56.475923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Request method failure captured: Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 HTTP/1.1 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2Ba2Da-ehBHDesIrcgDFA3lA1KgQrWEUFDIeCHqgOF9KhPcNdR6P2u1G8xQyRZGgAgjeXMgX_c5FhDetQdAHZ8Bt6WCqSXoS2rFn_mggGc7thSZvMlISdU0eg7gYUW8SiIF4C3j_SUhctUlKAg__5eNzle3kG2QmJW6mEl_ZctiuMq0 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:56.479083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}] Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: bc2f1a0d431341d6a5d1a8c5de131ece Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:10:56.480644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': 'e827c810-220b-4fe1-9d16-01bc8372b933', 'id': 'eaa89bcc-652b-4a03-941b-8440920143e6'} Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.481294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:10:56.482640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.484248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.485527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) Jul 20 15:10:56.486827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.488767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-20b4f3a6-d468-4e92-be07-26617b44c9d4) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:56.490161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a47bd9b1-d73a-40f0-b7b7-f0e821e49b47 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6" status: 400 len: 269 microversion: 2.1 time: 0.203275 Jul 20 15:10:56.491422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.492116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.499930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 568/1061] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:56 2022] DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933/os-volume_attachments/eaa89bcc-652b-4a03-941b-8440920143e6 => generated 269 bytes in 215 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:10:56.510578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.511638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.515983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.518797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.519644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.520076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.520214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.520630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.521551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.522530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.528939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:10:56.564054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc635e7d-684e-4e55-9718-038272d4a5a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061919 Jul 20 15:10:56.565179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 569/1062] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:56.571671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.572668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] [instance: 2f3521d1-81cb-4217-b43a-b84b94911839] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.579786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.580389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.581502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.581941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.597492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] HTTP exception thrown: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Request method failure captured: Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 HTTP/1.1 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BXL10Wwm98_XHeHgePYGKOZIaDO0bTWptr3MPLJgeBirRn1yA6S6l0igHvJ9EKX8dEBH3WHpsJO1IJL03MBCMt8_kE1Xre_rhFSdjjJ78xmi2Xu3cmk7e_k5XHWqespbbsMiwmY53mu74WKF3eA8WtsGEVZclFRhdVqi0l0UmUAlfo Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:10:56.599033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/2c8531e2e07a47eaae4c66d0b82df2d9"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/2c8531e2e07a47eaae4c66d0b82df2d9"}]}] Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 2c8531e2e07a47eaae4c66d0b82df2d9 Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestVolumeBootPattern-822710001 Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: dd610b017e7d4bd786db6bcd08e60722 Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestVolumeBootPattern-822710001-project Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.600760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '2f3521d1-81cb-4217-b43a-b84b94911839'} Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:10:56.601696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. Jul 20 15:10:56.602917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:10:56.604983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Returning 404 to user: Instance 2f3521d1-81cb-4217-b43a-b84b94911839 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:10:56.604983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44e2129e-582f-4c7d-ba71-44e5577bf06e tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839" status: 404 len: 111 microversion: 2.1 time: 0.034192 Jul 20 15:10:56.604983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 570/1063] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/2f3521d1-81cb-4217-b43a-b84b94911839 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:10:56.616172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:56.617223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:56.628595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.629562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.630835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.631360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.664312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:10:56.701327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:56.710298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce95d532-a91a-49ad-a9fe-b53e54d0e372 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.108083 Jul 20 15:10:56.710865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 571/1064] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:56 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:10:56.780166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-f2ae0126-325d-4a62-ba42-7f3bd8b8af67 req-d0b3e5a3-9781-4def-a38f-731c48256da8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f30068fe-59db-4a1f-9ad9-040b4e98f342", "name": "network-vif-deleted", "tag": "50cb07de-2875-4d87-a597-7c1823c2dd74"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:10:56.788352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f2ae0126-325d-4a62-ba42-7f3bd8b8af67 req-d0b3e5a3-9781-4def-a38f-731c48256da8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:56.788895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f2ae0126-325d-4a62-ba42-7f3bd8b8af67 req-d0b3e5a3-9781-4def-a38f-731c48256da8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:56.799554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-f2ae0126-325d-4a62-ba42-7f3bd8b8af67 req-d0b3e5a3-9781-4def-a38f-731c48256da8 service nova] Creating event network-vif-deleted:50cb07de-2875-4d87-a597-7c1823c2dd74 for instance f30068fe-59db-4a1f-9ad9-040b4e98f342 on hv-cinder-84902 Jul 20 15:10:56.804341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-f2ae0126-325d-4a62-ba42-7f3bd8b8af67 req-d0b3e5a3-9781-4def-a38f-731c48256da8 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092281 Jul 20 15:10:56.804883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 572/1065] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:10:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:57.294107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:57.295158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:57.310310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.311376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.312162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.312771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.351914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9645974c-ac9d-4c45-ba88-4f16426e9a42 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062994 Jul 20 15:10:57.352482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 573/1066] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:57 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:57.356978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:57.357883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:57.363921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.364393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.365140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.365571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.382715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:10:57.397915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0fa10ac-14c4-4821-ae23-897ec30c6910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044465 Jul 20 15:10:57.398578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 574/1067] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:57 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:57.579375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:57.580423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:57.588268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.588848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.589764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.590302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.623735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-41a9d14f-96a7-490e-9800-b62f56f71c58 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050846 Jul 20 15:10:57.624282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 575/1068] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:57.731333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:57.732334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:57.739554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.739998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.740750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:57.741180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:57.803517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:57.811869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1ccfed1-711a-41e0-8040-03484fe25027 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087054 Jul 20 15:10:57.812476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 576/1069] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:10:58.368906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:58.370388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:58.384133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.384883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.386144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.386859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.418750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-174db2db-e0c9-45e1-b2e9-a6be1b6d1fe6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057864 Jul 20 15:10:58.419363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 577/1070] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:58 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:58.423988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:58.424890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:58.430722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.431203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.431992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.432421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.462949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-945cd206-c67b-4bd7-891f-6d1429d85854 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042588 Jul 20 15:10:58.463535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 578/1071] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:58.639210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:58.640707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:58.647209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.647662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.648393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.648839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.679586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce548bba-1203-414e-b6d1-f92e659cb29b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047386 Jul 20 15:10:58.680138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 579/1072] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:58.832267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:58.833265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:58.843839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.844310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.845075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:58.845510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:58.912423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:10:58.924861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ed025350-02e0-4c18-a950-4f552be2e975 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098706 Jul 20 15:10:58.925645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 580/1073] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:58 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:10:59.436016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:59.437557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:59.446803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.447276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.448042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.448468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.482182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d284563c-62d7-4088-8d1a-7318e0b7e106 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053855 Jul 20 15:10:59.482826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 581/1074] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:59.487960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:59.489366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:59.498722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.499397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.500388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.500819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.530734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5dadf605-3729-418e-b64a-7e6be42ce030 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046957 Jul 20 15:10:59.531361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 582/1075] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:10:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:10:59.694278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:59.695425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:59.703669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.704262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.705304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.705929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.737987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-95cb3d26-bf6f-47ad-8895-329eacbae225 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048941 Jul 20 15:10:59.738529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 583/1076] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:10:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:10:59.950450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:10:59.951414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:10:59.958475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.958954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:10:59.959743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:10:59.960178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.029251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:00.037428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c0da533-2893-4fc3-ac97-4982b118f007 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093692 Jul 20 15:11:00.037993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 584/1077] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:10:59 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:00.498828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:00.500536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:00.503931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:00.508088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.508528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.509280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.509712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.540590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72a3ec0d-7db2-4830-976c-6fff9c787334 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048501 Jul 20 15:11:00.541138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 585/1078] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:00.545699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:00.546648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:00.552509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.552950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.553686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.554118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.582330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5d525200-b850-4ed5-b049-8f4fc8a07e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040228 Jul 20 15:11:00.582867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 586/1079] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:00.753775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:00.755101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:00.765745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.766195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.766950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:00.767414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:00.798180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6e71d6f9-5725-4a9a-bccd-b00ca2445ade tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051065 Jul 20 15:11:00.798730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 587/1080] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:01.061400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:01.062468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:01.069518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.069981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.070738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.071196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.139115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:01.150684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ec621a7a-a7cf-4bd4-bb59-1a812146ea56 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095429 Jul 20 15:11:01.151447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 588/1081] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:01 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:01.553989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:01.555160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:01.563123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.563706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.564647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.565178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.597565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ecb8e5a9-232f-4f97-97ba-fb2ff609fcc1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048635 Jul 20 15:11:01.598118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 589/1082] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:01 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:01.603421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:01.604373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:01.611407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.611826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.612587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.613019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.643512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0cd6771c-8850-4e59-af3e-cd673caf4cef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044360 Jul 20 15:11:01.644068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 590/1083] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:01.813572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:01.814589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:01.821608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.822136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.822968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:01.823479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:01.827637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:01.854770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-50438655-1a4a-4cfc-9b1d-8b2a05271b6d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047467 Jul 20 15:11:01.855467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 591/1084] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:02.173119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:02.173671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:02.180894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.181361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.182186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.182672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.248102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:02.260387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3c14b7e0-6b91-4d14-bd10-38e67ff95584 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092926 Jul 20 15:11:02.261136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 592/1085] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:02 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:02.611507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:02.612949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:02.622765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.623469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.624877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.625322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.655637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c55de18-c696-4db7-9c0b-01545b30e27b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049519 Jul 20 15:11:02.656300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 593/1086] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:02 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:02.661426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:02.662318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:02.668253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.668692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.669429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.669851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.698176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e74019a-3ef8-49bf-b804-3ba316e18a2e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040848 Jul 20 15:11:02.698787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 594/1087] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:02.870582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:02.872097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:02.882512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.883336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.884596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:02.885314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:02.917034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81cbbb63-1e69-4ded-aa6e-75ea4a9d1405 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053058 Jul 20 15:11:02.917583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 595/1088] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:03.282500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:03.283264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:03.291194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.291705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.292570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.293098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.359684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:03.367406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-67ea8aa8-c888-413f-83da-ec4789ea231d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090963 Jul 20 15:11:03.367995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 596/1089] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:03 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:03.671948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:03.672880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:03.679400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.679856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.680614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.681044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.712165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-21d0070a-c31b-4d2f-8870-aff742785593 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046888 Jul 20 15:11:03.712740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 597/1090] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:03.717318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:03.718220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:03.724484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.724932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.725684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.726109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.756022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a4e564cc-ae81-4c46-bbca-ddcc2e617c38 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042272 Jul 20 15:11:03.756581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 598/1091] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:03.934920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:03.936337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:03.946041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.946467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.947579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:03.948153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:03.987559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34ad3918-be46-429b-a53a-8ca42e504a5c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061291 Jul 20 15:11:03.988113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 599/1092] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:04.388686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:04.389678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:04.397043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.397484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.398250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.398681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.460498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:04.469344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bf178ea9-ccba-43ce-9ce6-82bcc324e687 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087476 Jul 20 15:11:04.469909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 600/1093] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:04 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:04.728283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:04.729608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:04.739906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.740654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.741914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.742628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.777903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e4c0fc71-486c-43af-bb18-6355531f0d3b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056372 Jul 20 15:11:04.778466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 601/1094] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:04.783108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:04.784036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:04.790072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.790519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.791315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:04.791744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:04.821169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2f4b685d-4bdb-4389-933a-d9622d0f1cce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041667 Jul 20 15:11:04.821746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 602/1095] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:05.001960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:05.002824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:05.009522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.009969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.010722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.011172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.040474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5377c7bc-a66c-4e00-8d40-33f6d880f0d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043980 Jul 20 15:11:05.041018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 603/1096] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:05.491000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:05.492488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:05.502152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.502602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.503401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.503836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.570239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:05.582088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f172a1a9-7b9d-4fb1-ba64-81a91e09eaac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097463 Jul 20 15:11:05.582870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 604/1097] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:05 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:05.794187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:05.795723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:05.802576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.803016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.803785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.804207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.834635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7e3ad28-b0cd-4ccb-857c-dd5024a19917 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047353 Jul 20 15:11:05.835217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 605/1098] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:05.839876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:05.840760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:05.846840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.847340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.848060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:05.848482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:05.878608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-04d38cb3-ff9f-4d46-ac04-4357449c8675 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042369 Jul 20 15:11:05.879164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 606/1099] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:06.056169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:06.057585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:06.067836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.068581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.069831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.070554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.098520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:06.103522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c4531673-e135-43a3-a748-43d772ce1fe9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053914 Jul 20 15:11:06.104103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 607/1100] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:06 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:06.496927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=60 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:06.602821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:06.603488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:06.611940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.612341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.613103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.613541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.669324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=6 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:06.679996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:06.688024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-806da3c1-2bb6-4a83-9c7d-1ac564deec1e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089436 Jul 20 15:11:06.688580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 608/1101] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:06 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:06.850058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:06.851581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:06.860909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.861354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.862163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.862520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.895240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-faef4545-b673-4b5b-a0ef-43cb49e3e8d2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051724 Jul 20 15:11:06.895803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 609/1102] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:06.900325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:06.901233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:06.907612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.908063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.908844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:06.909276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:06.945799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f596658-dc0b-4699-a43e-4b8b307b4b93 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048704 Jul 20 15:11:06.946664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 610/1103] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:07.120156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:07.121525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:07.129248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.129698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.130485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.130917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.160612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0e61ed9-b560-48ac-b79d-22c6a9e4c367 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047131 Jul 20 15:11:07.161160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 611/1104] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:07 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:07.708132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:07.709137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:07.719648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.720090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.720845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.721275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.797814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:07.807740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-46f7aedc-ff79-4c05-bf7f-e2d9cfecb15f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.106175 Jul 20 15:11:07.808359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 612/1105] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:07 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:07.911683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:07.913166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:07.923435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.924228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.925489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.926211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.959737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-db96026c-9f9a-40df-8665-a68fbc720c1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055182 Jul 20 15:11:07.960283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 613/1106] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:07 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:11:07.964677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:07.965568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:07.971409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.971857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:07.972605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:07.973042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.001243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5992fe02-f99e-4182-a001-d6e8f5b46b9e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040009 Jul 20 15:11:08.001785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 614/1107] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:08.175262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:08.176225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:08.184010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.184564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.185500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.186040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.220762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-28697860-736d-4a5b-8e46-8676afddcc6c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051263 Jul 20 15:11:08.221314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 615/1108] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:08.829520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:08.830168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:08.837540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.838014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.838849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.839342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.902218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:08.910046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84457dcf-2c88-4608-acb7-a8c8efaae32e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086345 Jul 20 15:11:08.910614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 616/1109] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:08 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:08.975131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:08.976664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:08.986151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.986598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:08.987399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:08.987842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.019352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74b06f64-c64f-40a2-b2df-8712f8407707 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050358 Jul 20 15:11:09.019910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 617/1110] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:08 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:09.024381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:09.025267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:09.030829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.031351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.032087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.032511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.061288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dbd4794e-43b8-47ab-a111-f7fed04a48d8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040399 Jul 20 15:11:09.061828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 618/1111] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:09.235761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:09.237103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:09.247303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.248082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.249310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.249946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.279756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87b5bd50-5cff-4236-92f2-13803be717a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050835 Jul 20 15:11:09.280298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 619/1112] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:09.931323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:09.932330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:09.941306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.941756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:09.942509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:09.942939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.011469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:10.019189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a085c593-cba5-4166-9317-f47cfe2ed50b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094475 Jul 20 15:11:10.019754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 620/1113] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:09 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:10.031064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:10.032002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:10.038134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.038551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.039358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.039769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.069468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f73f223-2339-423a-b78e-1e83c873ab0b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042173 Jul 20 15:11:10.070263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 621/1114] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:10 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:10.074788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:10.075703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:10.081302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.081746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.082487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.082917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.112831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-50c5db9a-8ee6-481b-b03e-f304164eb07f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041455 Jul 20 15:11:10.113386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 622/1115] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:10.293192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:10.294314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:10.301321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.301739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.302535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:10.302964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:10.335457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84827092-72f9-421b-a0b1-ff219311b9aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046851 Jul 20 15:11:10.336154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 623/1116] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:11.040593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:11.041560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:11.050514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.050958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.051737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.052169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.070185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:11.115392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:11.125594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba9cb249-c246-4709-aeef-16de6eeebf73 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091547 Jul 20 15:11:11.126191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 624/1117] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:11 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:11.130765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:11.132081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:11.139864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.140404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.141172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.141607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.171018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-daac89a5-8fd4-48ec-983c-041824b3b427 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043823 Jul 20 15:11:11.171965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 625/1118] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:11 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:11.176331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:11.177205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:11.183161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.183612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.184401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.184810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.213553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-04418dc2-4015-44fc-803b-07b621134619 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040629 Jul 20 15:11:11.214111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 626/1119] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:11.351784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:11.353326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:11.361768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.362344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.363104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:11.363570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:11.393571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d46ffc9b-4cb8-40d0-9a7e-71a91f7f46bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048082 Jul 20 15:11:11.394052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 627/1120] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:12.146403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:12.147568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:12.158397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.158973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.159832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.160287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.226124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:12.234367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-01392680-fe8a-4fee-a927-3c042687dd0b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094875 Jul 20 15:11:12.234968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 628/1121] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:12 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:12.239663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:12.240585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:12.246173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.246656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.247465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.247928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.276744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4f63f511-5c7e-4c94-b2e8-596cdfcd3c1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040650 Jul 20 15:11:12.277279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 629/1122] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:12 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:12.281990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:12.282881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:12.288752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.289200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.289930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.290376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.320047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-90f5dbeb-db38-4abb-a52a-eac8e24c69c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041777 Jul 20 15:11:12.320597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 630/1123] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:12.408176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:12.409216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:12.416750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.417279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.418177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:12.418679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:12.454216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4aa6d6db-e42c-4fda-984a-407d848496dc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051772 Jul 20 15:11:12.455604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 631/1124] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:13.253218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:13.253902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:13.260956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.261418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.262173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.262595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.325591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:13.333670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9b2da9c-d71e-4779-ae32-734dc186fdde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.084977 Jul 20 15:11:13.334220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 632/1125] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:13 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:13.338803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:13.339725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:13.346317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.346759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.347528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.347972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.376894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbe93403-2e71-47c0-8971-d3a9fd5b70d3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041672 Jul 20 15:11:13.377438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 633/1126] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:13.382337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:13.383287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:13.388962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.389400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.390206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.390633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.419025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-530a1a20-e90d-467d-a398-d6fc18c0f466 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040568 Jul 20 15:11:13.419604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 634/1127] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:13.466359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:13.467390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:13.470436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:13.474308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.474750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.475574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:13.476000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:13.507399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-88279dfa-e7ad-4f28-84e3-385aa8298280 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044487 Jul 20 15:11:13.507876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 635/1128] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:14.354444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:14.355629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:14.364542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.364991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.365747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.366176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.430103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:14.440552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a918a74-60e8-4503-abc9-bbebfca6c0f5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092455 Jul 20 15:11:14.441117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 636/1129] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:14 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:14.445620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:14.447167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:14.453209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.453666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.454421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.454851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.484686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e3e09998-735d-42d9-b089-2db2ddbefa91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042536 Jul 20 15:11:14.485368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 637/1130] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:14.491428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:14.492926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:14.502907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.503388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.504150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.504578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.533963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1df5f077-ca01-4b7c-bff3-c1f0be31ea00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047736 Jul 20 15:11:14.534512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 638/1131] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:14.538881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:14.539804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:14.545663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.546130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.546879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:14.547344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:14.575521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-768e86a5-5b05-440a-b043-e0be32e11e97 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040019 Jul 20 15:11:14.576058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 639/1132] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:15.460985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:15.462085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:15.471597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.472041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.472806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.473257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.536879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:15.547890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-29fef930-68a9-4aa0-acb3-207acaa0be99 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093283 Jul 20 15:11:15.548470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 640/1133] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:15 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:15.553201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:15.554099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:15.560091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.560536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.561291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.561725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.565439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:15.594865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59151c3f-87ee-42fb-8795-d47f745b032b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044964 Jul 20 15:11:15.595718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 641/1134] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:15.600668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:15.601976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:15.610040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.610486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.611269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.611673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.641456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-58d47e3b-f578-4e9a-93e9-b42f91b61bd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044725 Jul 20 15:11:15.642007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 642/1135] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:15.646591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:15.647521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:15.653352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.653793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.654509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:15.654981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:15.683327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2d043ad0-685d-4370-badd-3fe8fb9d6655 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040318 Jul 20 15:11:15.683874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 643/1136] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:15 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:16.571721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:16.572718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:16.581505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.581948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.582704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.583167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.647622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:16.655384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45de0454-beae-496a-a68b-5cf4f6b83717 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090186 Jul 20 15:11:16.656002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 644/1137] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:16 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:16.660454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:16.661433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:16.667417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.667865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.668622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.669032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.698729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ed72beaa-2847-4dfa-af5d-e4ef793d06db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041692 Jul 20 15:11:16.699323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 645/1138] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:16.703854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:16.704741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:16.710568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.711013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.711791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.712222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.741527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c2cab8f3-0f8c-4cfc-bc0a-a303fe3e7dcc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041158 Jul 20 15:11:16.741995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 646/1139] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:16.746344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:16.747262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:16.753026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.753468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.754237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:16.754639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:16.784517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5c3b684c-fcad-43f8-b54c-a5042e30aafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041571 Jul 20 15:11:16.785037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 647/1140] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:17.676262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:17.677290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:17.687939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.688378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.689128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.689553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.751949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:17.763796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-970318a1-082f-4aee-9305-ae6ef7bef45f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093716 Jul 20 15:11:17.764433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 648/1141] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:17 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:17.769217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:17.770185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:17.776692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.777149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.777939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.778373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.809018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dc7824f1-7afa-43b9-9f9e-c0fdfdbbe7da tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043572 Jul 20 15:11:17.809572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 649/1142] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:17 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:17.814173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:17.815082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:17.821088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.821551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.822338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.822770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.852838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4245f32e-bdd9-4553-92f9-afbe80d774ef tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042265 Jul 20 15:11:17.853374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 650/1143] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:17.857745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:17.858709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:17.864693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.865140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.865891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:17.866323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:17.895764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2f5bfaf4-2fb3-4128-8ef5-658dfb2db1d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041395 Jul 20 15:11:17.896340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 651/1144] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:18.786478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:18.787563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:18.797636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.798091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.798854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.799316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.860382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:18.868478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ac23040f-e79a-4f9f-ab65-008e7b2cd329 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088786 Jul 20 15:11:18.869045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 652/1145] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:18 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:18.873818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:18.874740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:18.880841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.881281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.882347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.882778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.913010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c6edd817-ca7c-45bc-b7ec-5cf9aab458ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042949 Jul 20 15:11:18.913569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 653/1146] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:18 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:18.918265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:18.919196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:18.924859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.925303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.926029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.926462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.954766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-60b11084-7165-40c6-b49a-ecc5c427b637 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040241 Jul 20 15:11:18.958697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 654/1147] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:18.959868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:18.960750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:18.966177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.966620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.967394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:18.967823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:18.997132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d75b4d35-6566-48a2-9bf1-330f368287de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040605 Jul 20 15:11:18.997895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 655/1148] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:19.890105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:19.891216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:19.901974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:19.902711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:19.904023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:19.904741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:19.969389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:19.980673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5c433abc-b493-4e8d-b71f-a46b68b95c22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097314 Jul 20 15:11:19.981416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 656/1149] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:19 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:19.985856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:19.986757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:19.993774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:19.994344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:19.995397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:19.995946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:20.022014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:20.028052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e3f8c72d-2a0c-414a-8a11-99df923a5b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045651 Jul 20 15:11:20.028651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 657/1150] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:20.033364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:20.034252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:20.040471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:20.040913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:20.041652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:20.042305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:20.073013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4098c680-aa28-418e-95ab-a5c9854e5f1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043424 Jul 20 15:11:20.073569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 658/1151] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:20.078201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:20.079118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:20.084726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:20.085178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:20.085932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:20.086370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:20.115670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cda8b490-0d3a-448c-b1b1-de5d4f8feb37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041044 Jul 20 15:11:20.116221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 659/1152] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:21.002541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:21.003653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:21.015078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.015551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.016323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.016757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.093994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:21.106454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d07292f-ad63-4dab-ba56-f815b2572233 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.110993 Jul 20 15:11:21.107442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 660/1153] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:20 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:21.112046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:21.112966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:21.118879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.119366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.120127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.120582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.150122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-004b29a5-92ea-4b07-8dc1-dd7c2769f23d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041584 Jul 20 15:11:21.150592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 661/1154] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:21.155032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:21.155996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:21.161725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.162166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.162917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.163381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.191889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6e7d5c42-7e5a-4236-b6a6-83dfb7fcba19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040318 Jul 20 15:11:21.192425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 662/1155] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:21.196804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:21.197703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:21.203441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.203889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.204637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:21.205066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:21.234061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3ef88ecb-5ddc-4142-ba01-a1a7658bf75e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040668 Jul 20 15:11:21.234606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 663/1156] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:21 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:22.126549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:22.127341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:22.136051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.136632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.137585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.138131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.202789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:22.210616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be5f3d94-4544-4c04-9c03-dc941dd0d877 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089929 Jul 20 15:11:22.211207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 664/1157] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:22 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:22.215690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:22.216604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:22.222403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.222849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.223635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.224071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.255820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7c5c0b8-c347-4afe-8470-5e327ca09b61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043598 Jul 20 15:11:22.256345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 665/1158] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:22 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:22.261097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:22.261984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:22.267752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.268204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.268953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.269382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.298103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6d5a93f0-b569-49b4-b7ff-269f0b187b3c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040449 Jul 20 15:11:22.298642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 666/1159] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:22.302987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:22.303895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:22.309914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.310543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.311587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:22.312124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:22.340300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-890e761d-6c66-46e4-a97e-feb31741d06f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040647 Jul 20 15:11:22.340837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 667/1160] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:23.232115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:23.233104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:23.240419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.240872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.241659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.242087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.303731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:23.312233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebe9870e-eca5-4b47-a404-8495c858ee21 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087026 Jul 20 15:11:23.312798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 668/1161] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:23 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:23.317307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:23.318247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:23.324245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.324695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.325468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.325877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.356424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e3b67e91-7964-4d05-b5f2-082d920f5a4f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042496 Jul 20 15:11:23.357878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 669/1162] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:23.362884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:23.363870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:23.370014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.370489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.371320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.371758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.405380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56a799e7-0dac-4181-b82d-2c60773446af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046645 Jul 20 15:11:23.406057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 670/1163] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:23.410482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:23.411426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:23.417121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.417569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.418319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:23.418751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:23.447493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-161175a8-eccc-4cbe-bd8f-92e6c11235ae tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040417 Jul 20 15:11:23.448050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 671/1164] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:23.711314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:11:23.711666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:11:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:11:24.333587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:24.334649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:24.344772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.345197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.345981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.346415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.417515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:24.425672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9496829f-75ed-4a15-b203-fbfeed2e4b59 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098713 Jul 20 15:11:24.426233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 672/1165] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:24 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:24.430752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:24.431746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:24.437521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.437970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.438727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.439185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.469192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c7c974aa-09c1-4684-9cca-4cb27727bb91 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041983 Jul 20 15:11:24.469741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 673/1166] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:24.474297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:24.475221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:24.481586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.481865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.482622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.483084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.499801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:24.513327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c52921f5-2902-4252-b18d-85a0c6c6481b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042599 Jul 20 15:11:24.513874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 674/1167] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:24.518478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:24.519383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:24.524993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.525431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.526146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:24.526699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:24.559089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b1deb713-953d-4a28-b58f-74bb5aee0ec9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043890 Jul 20 15:11:24.559773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 675/1168] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:25.446655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:25.447780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:25.456364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.456808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.457563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.457994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.522915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:25.531300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e23ceb21-0c14-434d-b624-c3cfbe1eafb5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091079 Jul 20 15:11:25.531870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 676/1169] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:25 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:25.536369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:25.537288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:25.543273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.543725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.544489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.544924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.574228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2d108dda-1750-4684-b103-bb2a02cbe063 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041359 Jul 20 15:11:25.574784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 677/1170] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:25.579291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:25.580186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:25.585776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.586222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.586971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.587430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.615906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9dd5a22f-3a00-4e0a-b7d5-66510c339351 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040118 Jul 20 15:11:25.616458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 678/1171] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:25.620802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:25.621690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:25.629057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.629659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.630817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:25.631826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:25.663427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-73275531-f15f-4f3b-9813-cb033e29b2e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045977 Jul 20 15:11:25.663981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 679/1172] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:26.551755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:26.552677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:26.562634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.563093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.563879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.564310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.628523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:26.636805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebfc2373-dd7a-44f9-a12d-eace6259a535 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091009 Jul 20 15:11:26.637371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 680/1173] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:26 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:26.642034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:26.642950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:26.648728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.649177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.649931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.650363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.679180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9a1b103-6dac-4ba4-97e3-2dcbcc268272 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040696 Jul 20 15:11:26.679725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 681/1174] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:26.684305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:26.685196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:26.690941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.691419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.692165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.692594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.729955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be4b2bf0-33fb-4a76-a124-943660421c25 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049178 Jul 20 15:11:26.730505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 682/1175] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:26.734941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:26.735962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:26.741744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.742192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.742944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:26.743417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:26.772752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1bd497e0-b871-4dd9-b5ff-c1cbc6648d17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041270 Jul 20 15:11:26.773323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 683/1176] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:27.657804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:27.658796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:27.669717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.670455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.671609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.672033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.736308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:27.747948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-baa30663-5d84-4693-ac79-358c632142f9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096259 Jul 20 15:11:27.748554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 684/1177] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:27 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:27.753339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:27.754273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:27.760135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:27.762725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.763494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.764846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.765284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.796278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ae7474df-2a69-414e-994a-c27ffa0c2502 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046599 Jul 20 15:11:27.796851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 685/1178] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:27 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:27.801372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:27.802294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:27.808289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.808737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.809497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.809930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.838741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7dd1986d-342c-49ee-bdaf-d7603a772a11 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040892 Jul 20 15:11:27.839342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 686/1179] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:27.843795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:27.844676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:27.850806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.851297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.852054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:27.852483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:27.888232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7795ee83-e598-4874-8495-7514a0dc4d87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047606 Jul 20 15:11:27.888890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 687/1180] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:27 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:28.769623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:28.770657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:28.781053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.781499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.782278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.782681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.847037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:28.855079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe4522b9-9bed-4ca3-af98-53d7676b291a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092053 Jul 20 15:11:28.855743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 688/1181] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:28 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:28.860224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:28.861136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:28.867080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.867572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.868336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.868769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.902970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-73872629-d970-45e3-aeb0-98bedc39d1b9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045824 Jul 20 15:11:28.903769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 689/1182] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:28.908316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:28.909206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:28.915154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.915599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.916347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.916827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.945965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1bc903c4-93f6-4ebf-83e9-c2dd071e6a3d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041183 Jul 20 15:11:28.946493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 690/1183] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:28.950845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:28.951750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:28.957613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.958048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.958781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:28.959249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:28.962724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:28.988771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be733f79-6fcf-4cbc-9771-5029c9a2b3f8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041307 Jul 20 15:11:28.989323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 691/1184] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:29.872560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:29.872560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:29.880007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:29.880462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:29.881597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:29.882050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:29.948053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:29.956572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-68859f36-f526-4dc9-a356-bab122453bd0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088048 Jul 20 15:11:29.957142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 692/1185] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:29 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:29.961979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:29.962894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:29.969234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:29.969683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:29.970435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:29.970865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.002046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9baf8dd7-0ccd-478e-b88b-ece811b75333 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043801 Jul 20 15:11:30.002581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 693/1186] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:30.007508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:30.008395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:30.014335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.014904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.015685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.016113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.044660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7fadaf03-a9fc-4efe-9969-6bd2f632f2a6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041015 Jul 20 15:11:30.045197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 694/1187] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:30.049677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:30.050591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:30.056197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.056634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.057378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.057803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.087075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6922c7eb-8a80-4616-9f2c-4b40c83c065f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040912 Jul 20 15:11:30.087607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 695/1188] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:30.977341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:30.978405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:30.989329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.990078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:30.991484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:30.992163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.058570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:31.069620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b8495049-0188-499d-aa1f-7fbc64a31c2f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098983 Jul 20 15:11:31.070183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 696/1189] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:30 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:31.074839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:31.075780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:31.081645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.082085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.082835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.083296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.112270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-144d9154-16c8-4cad-9c30-51fa78c684d0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041009 Jul 20 15:11:31.112819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 697/1190] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:31.117439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:31.118325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:31.124207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.124648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.125397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.125826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.154250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-95e43eb7-2225-4773-baf9-2f0ae798b3f2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040360 Jul 20 15:11:31.154731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 698/1191] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:31.159311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:31.160187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:31.165832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.166271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.166991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:31.167478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:31.195736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0d36c158-d53e-481e-a32e-0dffb612d82d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039949 Jul 20 15:11:31.196267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 699/1192] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:32.090522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:32.091653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:32.102484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.103280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.104536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.105252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.167703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:32.175509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d17c48b1-375e-4dea-ac27-375217acba22 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091346 Jul 20 15:11:32.176073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 700/1193] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:32 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:32.180742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:32.181668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:32.187499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.187946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.188697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.189121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.217398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6001917c-8cb2-4318-b0f6-3af70813126a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040265 Jul 20 15:11:32.217946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 701/1194] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:32.222465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:32.223384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:32.229061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.229506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.230244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.230669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.259219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3ea34258-7d39-47b7-9a78-c9c290f1ffab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040222 Jul 20 15:11:32.259759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 702/1195] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:32.264226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:32.265226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:32.270916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.271385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.272125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:32.272557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:32.300460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c0f714ce-7a5b-48c3-99d2-cd196d015877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039762 Jul 20 15:11:32.300996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 703/1196] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:32.995828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:11:32.996131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:11:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:11:33.196586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:33.197598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:33.208390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.209137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.210392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.211111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.277068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:33.285030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87ed830f-c61e-4fc7-8ff9-4d8a48c1e791 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095033 Jul 20 15:11:33.285633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 704/1197] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:33 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:33.290151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:33.291086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:33.296865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.297313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.298120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.298526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.328432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e406365b-a712-4ea6-ae14-6e8744a28cef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041754 Jul 20 15:11:33.328972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 705/1198] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:33 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:33.333442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:33.334352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:33.340114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.340556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.341300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.341755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.373313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-090ad9c7-2469-434d-b1ba-19ed759f2e19 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043342 Jul 20 15:11:33.373921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 706/1199] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:33.378310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:33.379290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:33.385569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.386051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.386797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:33.387276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:33.411611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:33.416247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4dbc781-113e-49a2-a53f-981e0f373449 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041353 Jul 20 15:11:33.416781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 707/1200] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:34.306787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:34.307903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:34.318652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.319464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.320709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.321424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.390755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:34.398787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-08294600-7788-4e4f-83e6-285df81eb7c7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098953 Jul 20 15:11:34.399382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 708/1201] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:34 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:34.404034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:34.404933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:34.410604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.411040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.411811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.412234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.442921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3ccc15c-78b8-49f4-8895-3f7635ca7606 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042469 Jul 20 15:11:34.443550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 709/1202] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:34.448391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:34.449358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:34.455584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.456035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.456787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.457194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.489668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0602030-6e9f-4394-83be-613d8a971000 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044992 Jul 20 15:11:34.490228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 710/1203] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:34.494927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:34.495866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:34.501693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.502180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.502954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:34.503420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:34.532465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbeb9117-eb6a-4cd5-afbd-1ee10685fae1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041262 Jul 20 15:11:34.532975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 711/1204] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:35.419662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:35.420524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:35.431144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.431898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.433365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.434026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.496468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:35.505247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-809fbd63-9ed9-4efd-9b2c-035c3286c2aa tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092249 Jul 20 15:11:35.505864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 712/1205] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:35 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:35.510718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:35.511656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:35.517587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.518035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.518798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.519259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.548070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e92c8847-4240-40e4-9c2c-923e7b79b68b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040837 Jul 20 15:11:35.548614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 713/1206] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:35.553206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:35.554131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:35.559887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.560363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.561112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.561589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.589871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-887c5bd6-24e6-4f44-8ce7-ec4b4e3e0bce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040272 Jul 20 15:11:35.590441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 714/1207] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:35.595026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:35.595887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:35.601698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.602191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.602965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:35.603463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:35.633366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bfbaff53-bf92-47fd-bfbe-8cb335444438 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041936 Jul 20 15:11:35.633895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 715/1208] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:36.524105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:36.524814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:36.532473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.532911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.533656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.534082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.595041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:36.602668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a13a144b-967c-4bfd-8d81-29090c9afef8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.083848 Jul 20 15:11:36.603252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 716/1209] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:36 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:36.607918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:36.608875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:36.614516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.614950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.615727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.616146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.645078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eff38247-84d7-48be-b121-a56ffeb6618c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040856 Jul 20 15:11:36.645616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 717/1210] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:36 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:36.649958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:36.650831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:36.656605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.657038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.657770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.658189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.686678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ac1f5d10-3332-4837-a0a3-1b08cac7d87f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040094 Jul 20 15:11:36.687268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 718/1211] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:36.691555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:36.692428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:36.698143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.698590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.699355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:36.699850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:36.728553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a718caef-84f6-49cf-85a4-7f0c22f85b77 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040339 Jul 20 15:11:36.729237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 719/1212] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:37.625004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:37.625986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:37.637294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.638038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.639365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.640018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.706872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:37.714678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef93f464-11d7-4521-80d4-a8dc85b47ce7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096623 Jul 20 15:11:37.715343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 720/1213] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:37 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:37.720057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:37.720971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:37.726994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.727477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.728276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.728737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.761101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a922c00a-6679-46cc-9a59-0cdf031a36a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044700 Jul 20 15:11:37.761752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 721/1214] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:37 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:37.766519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:37.767449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:37.773257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.773703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.774456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.774890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.804090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-242d1a66-9066-4690-9bda-b24da13f62ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041282 Jul 20 15:11:37.804650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 722/1215] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:37.809062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:37.809949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:37.816032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.816466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.817216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:37.817643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:37.846833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36c8964f-d04b-4430-b874-d0e8d4a79f71 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041213 Jul 20 15:11:37.847478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 723/1216] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:37 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:38.734903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:38.735976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:38.746503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.747304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.748585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.749217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.767676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:38.818021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:38.826297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5c22050b-5e9d-48d6-b385-29ae06d00b16 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097788 Jul 20 15:11:38.826867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 724/1217] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:38 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:38.831905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:38.832818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:38.839187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.839630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.840388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.840813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.871488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-785836a4-c4b0-4819-a27c-07217065f994 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043494 Jul 20 15:11:38.872038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 725/1218] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:38 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:38.876493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:38.877411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:38.883702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.884141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.884889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.885311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.914480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-670bb724-b5c3-4cac-88ec-5312a18fb661 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041451 Jul 20 15:11:38.915032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 726/1219] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:38.919435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:38.920323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:38.925997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.926439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.927220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:38.927657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:38.955758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e3a29fd-bd94-4c6d-8bf5-2d33374372c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039752 Jul 20 15:11:38.956310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 727/1220] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:39.848068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:39.848660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:39.855896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.856429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:39.857220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.857696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:39.920635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:39.928418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-895f159d-326f-4b9e-9277-0a8e7078e56b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.084304 Jul 20 15:11:39.928963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 728/1221] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:39 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:39.933522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:39.934450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:39.940267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.940710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:39.941455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.941878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:39.970083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-801d53b7-5d84-4056-a076-39e2ce79865b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040157 Jul 20 15:11:39.970619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 729/1222] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:39.974951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:39.975861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:39.981419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.981885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:39.982614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:39.983036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:40.011267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d4af6e32-9df7-41a1-8c45-db0faedbe884 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039617 Jul 20 15:11:40.011912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 730/1223] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:40.016158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:40.017031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:40.022596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:40.023033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:40.023792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:40.024220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:40.052057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a64bb0f9-aeef-4a12-8c68-866761d3a44a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039147 Jul 20 15:11:40.052508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 731/1224] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:40 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:40.949363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:40.950318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:40.961253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:40.961996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:40.963305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:40.964029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.027516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:41.035584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5d843db1-85f0-4de2-9d85-e1294036856f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092661 Jul 20 15:11:41.036150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 732/1225] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:40 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:41.040658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:41.041592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:41.047542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.047991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.048746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.049228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.078290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-403fbd01-7296-4c45-9d81-4f79b95e5264 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041149 Jul 20 15:11:41.078806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 733/1226] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:41 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:41.083266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:41.084166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:41.089910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.090352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.091139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.091571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.120922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7fcb6577-7b00-4d2c-b5fb-b62064e70a6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041079 Jul 20 15:11:41.121480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 734/1227] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:41.125916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:41.126802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:41.129414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:41.133270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.133716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.134476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:41.134907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:41.164475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-662a86b5-83d5-4c9a-ae0c-6f171930ad60 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041952 Jul 20 15:11:41.164947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 735/1228] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:42.056724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:42.057920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:42.068845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.069581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.070811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.071634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.135959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:42.143930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c87f9e4-8eee-4db1-888d-55381d1e0dea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093604 Jul 20 15:11:42.144483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 736/1229] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:42 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:42.148931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:42.149852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:42.155641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.156080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.156824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.157243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.187141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f735bd4-f329-45bd-a58f-e64d018c5971 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041579 Jul 20 15:11:42.187680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 737/1230] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:42.192433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:42.193319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:42.199035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.199502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.200244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.200665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.230597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-097ed70d-0623-40e0-a0fd-fd751a246f50 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041878 Jul 20 15:11:42.231215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 738/1231] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:42.235856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:42.236777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:42.242806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.243291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.244047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:42.244504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:42.274211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cc422ccd-4ea3-4612-95ac-284a1c516fb2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041966 Jul 20 15:11:42.274720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 739/1232] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:43.165992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:43.167020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:43.178507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.179323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.180584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.181304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.248848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:43.257245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c1b7d3fa-115c-4b05-bf64-5378371308fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098506 Jul 20 15:11:43.257815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 740/1233] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:43 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:43.262380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:43.263355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:43.269626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.270102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.270911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.271398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.275093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:43.301810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7a49da6-e05d-4123-b2fe-59b7f221d7a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042993 Jul 20 15:11:43.302362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 741/1234] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:43.307014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:43.307985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:43.313759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.314204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.314951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.315420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.346155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f038115-42ed-43ad-83f3-c6ca126b5ae9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042800 Jul 20 15:11:43.350125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 742/1235] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:43.351467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:43.352455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:43.359309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.359774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.360601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:43.361040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:43.391635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f46eb0d3-7273-42da-8730-7233f826b20c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043793 Jul 20 15:11:43.392212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 743/1236] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:44.276666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:44.277897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:44.293657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.294747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.296368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.297034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.372153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:44.384675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a3751f8b-2784-47a8-bbb4-dc7b46a534eb tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.113928 Jul 20 15:11:44.385465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 744/1237] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:44 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:44.390210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:44.391179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:44.397107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.397551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.398466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.398925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.430025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e97552d-1c26-4f53-8a65-8216a41dfd64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043581 Jul 20 15:11:44.430579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 745/1238] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:44 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:44.435245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:44.436268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:44.442274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.442719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.443503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.443941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.473749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cf823d4e-468d-464c-ac4f-92ff59590428 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042100 Jul 20 15:11:44.474295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 746/1239] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:44.479172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:44.482864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:44.486479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.486948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.487737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:44.488168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:44.520079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5de2f4e5-35d8-4103-8157-0882891aa476 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044632 Jul 20 15:11:44.520961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 747/1240] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:45.401955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:45.402619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:45.409256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.409704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.410463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.410892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.473711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:45.490223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c7b2b09-b14f-405e-aa22-b8ebe6183d03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092149 Jul 20 15:11:45.490807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 748/1241] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:45 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:45.495513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:45.496512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:45.503331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.504058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.504822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.505249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.535682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cd834e0d-a348-4053-9055-77d62d940f1d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043815 Jul 20 15:11:45.536202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 749/1242] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:45.540599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:45.541484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:45.547092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.547539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.548275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.548701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.576780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-843e79e9-42e3-4ea8-b6a8-e0c91273e047 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039543 Jul 20 15:11:45.580625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 750/1243] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:45.581899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:45.582783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:45.588669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.589101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.589844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:45.590275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:45.618441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6219e67e-59b4-4c2c-835f-d93fbab5eeaa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040034 Jul 20 15:11:45.618973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 751/1244] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:46.511548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:46.512159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:46.519365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.519811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.520555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.520980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.582695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:46.594925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b655962f-1bf7-4ace-962e-9c895702ef14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089587 Jul 20 15:11:46.595842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 752/1245] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:46 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:46.602975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:46.604406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:46.611458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.611992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.612758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.613189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.642830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40d7e347-99db-4df5-bf89-f92593306ffb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045748 Jul 20 15:11:46.643415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 753/1246] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:46 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:46.648121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:46.649067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:46.655068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.655539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.656289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.656727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.688291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7e72cb7-8709-42ed-947c-fd0384f3e240 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043734 Jul 20 15:11:46.688844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 754/1247] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:46.693299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:46.694187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:46.700062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.700514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.701260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:46.701721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:46.730582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4239174c-015f-4a2b-b5ce-9a388ab8949b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040732 Jul 20 15:11:46.731123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 755/1248] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:47.620365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:47.621486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:47.629143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.629580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.630323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.630747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.695969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:47.708691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2c372d6a-fec7-4bd1-8953-1c0e363b0875 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094451 Jul 20 15:11:47.709494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 756/1249] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:47 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:47.716628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:47.717886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:47.723853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.724301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.725087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.725524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.750632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:47.755350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b57ceb0b-f826-46ba-93ac-4ac12dbb8b79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044761 Jul 20 15:11:47.756225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 757/1250] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:47 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:47.760949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:47.761834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:47.767859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.768306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.769059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.769494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.799080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8374ca51-40ef-4e4f-8408-77c67134e5f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041850 Jul 20 15:11:47.799657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 758/1251] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:47.804014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:47.804923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:47.810505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.810947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.811761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:47.812188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:47.841077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-69858af6-4176-44b5-a7e6-0f6d52a62a54 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040452 Jul 20 15:11:47.841620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 759/1252] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:48.733490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:48.734244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:48.741841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.742260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.743087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.743598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.806671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:48.814779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4641720d-f9a1-4cba-806b-8270fc6c9efd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088229 Jul 20 15:11:48.815456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 760/1253] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:48 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:48.824442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:48.825825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:48.832330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.832778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.833544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.833949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.862932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c09670a2-a751-454d-bd0d-26d3d8ac6d1b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046056 Jul 20 15:11:48.863532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 761/1254] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:48 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:48.868030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:48.868930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:48.874881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.875366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.876122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.876556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.905615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fb09074a-56c2-4c31-b7c5-625046c4c701 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041048 Jul 20 15:11:48.906174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 762/1255] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:48.910543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:48.911462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:48.917373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.917825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.918577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:48.919015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:48.954078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff6cb9e6-0b34-43ac-b60c-379520dbde06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046639 Jul 20 15:11:48.954770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 763/1256] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:49.835780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:49.836593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:49.843507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.843947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:49.844710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.845136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:49.918740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:49.926860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7c1c10a-da83-4492-9a2e-955ab54d2831 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095723 Jul 20 15:11:49.927464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 764/1257] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:49 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:11:49.932024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:49.932931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:49.939164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.939643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:49.940399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.940828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:49.970211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-179c3f14-1d95-4bea-9d46-e1aae4490062 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041731 Jul 20 15:11:49.970764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 765/1258] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:49 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:49.975240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:49.976133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:49.981940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.982412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:49.983228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:49.983658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:50.013365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f00ef69f-972d-4eee-9fa8-fe2e54c4131e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041602 Jul 20 15:11:50.013881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 766/1259] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:50.018286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:50.019190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:50.024992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:50.025436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:50.026150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:50.026601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:50.055535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6dcff8e0-d480-40f7-8693-6ee5d27ec38d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040617 Jul 20 15:11:50.056066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 767/1260] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:50.947218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:50.948018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:50.956351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:50.956795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:50.957553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:50.957983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.028492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:51.036414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4ea759e9-ae6b-4317-9ea0-af8e77ce4376 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095540 Jul 20 15:11:51.036980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 768/1261] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:50 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:51.041572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:51.042474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:51.048531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.048986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.049738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.050166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.079856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-309ee6fb-edfe-47ca-836f-050cbdb642ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041873 Jul 20 15:11:51.080406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 769/1262] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:51 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:51.084958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:51.085834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:51.091713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.092162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.092898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.093325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.123889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-46384edd-7603-4d9c-9cfa-8eb4564080ae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042339 Jul 20 15:11:51.124443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 770/1263] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:51.199574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:51.200525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:51.207085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.207521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.208270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:51.208694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:51.239008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b23fc5c-8859-4011-a3c6-89b5597d734c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.113594 Jul 20 15:11:51.239587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 771/1264] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:52.056986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:52.058122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:52.066178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.066603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.067449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.067883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.131565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:11:52.139715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4c5523c6-614a-4291-b265-cccc3c279704 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089168 Jul 20 15:11:52.140281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 772/1265] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:11:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:11:52.144757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:52.145664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:52.151753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.152198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.152956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.153386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.181830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c08088ae-445d-4926-a34d-671487c1c5ef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040559 Jul 20 15:11:52.182385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 773/1266] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:52 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:11:52.187086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:52.187986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:52.193843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.194288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.195034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.195496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.212763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:11:52.225950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bcace136-67df-40ce-986a-6aa3a517678d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042643 Jul 20 15:11:52.226535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 774/1267] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:11:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:11:52.251570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:52.252487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:52.258643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.259081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.259868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:52.260333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:52.289328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e89c4a05-2855-402e-a7db-343a8a11913e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041467 Jul 20 15:11:52.289846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 775/1268] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:11:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:11:53.232398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:11:53.233052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:11:53.239353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:53.239795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:11:53.240545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:11:53.240971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:03.236316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=82 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:03.260660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=12 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:12:51.681926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:12:51.684228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:12:51.685751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:12:51.686449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:12:51.687662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:51.689276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:12:51.690379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) Jul 20 15:12:51.691694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:12:51.693357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7c829bb8-3b54-4c4c-9296-bbc6ace2296d) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:12:51.693357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ed1e8153-d499-42ee-8a9a-586ef990b586 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 115.209898 Jul 20 15:12:51.693357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 494/1269] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:10:56 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 115211 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:12:51.693357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:51.693965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:51.699391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.699856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.700668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.701107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.763227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:12:51.771106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a58e4416-42d2-4b3d-8b8f-b82f6ddbae52 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.083807 Jul 20 15:12:51.771674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 495/1270] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:12:51 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:51.840445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:51.841485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:51.847494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.847933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.848684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.849106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.878062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bf19e1e6-a4c0-43e6-82da-5eb29350950c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.105378 Jul 20 15:12:51.882415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 496/1271] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:51 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:51.883777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:51.884445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:12:51.884930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:51.890545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.890997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.891794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.892228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.921997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3bc43cbe-7c03-4d3d-8e88-75934bfd199a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042003 Jul 20 15:12:51.923694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 497/1272] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:12:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:12:51.929101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:51.930027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:51.936204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.936655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.937422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:51.937859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:51.966836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-780ada35-2062-41de-856b-fe372dcbe674 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041486 Jul 20 15:12:51.967676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 498/1273] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:52.792709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:52.793729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:52.804327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.805069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.806320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.807079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.870656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:12:52.878984 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-28e28387-520a-47ab-ab64-fc725ec96dca tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092948 Jul 20 15:12:52.879903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 499/1274] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:12:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:52.889378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:52.890294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:52.896146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.896589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.897347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.897777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.928972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3549e837-e88b-4e61-b0ff-d44331e99baa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043115 Jul 20 15:12:52.929721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 500/1275] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:52 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:12:52.934055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:52.935167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:52.941161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.941609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.942370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.942817 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.972062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c9e342de-4be2-42e0-902e-cd89bb00949c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040617 Jul 20 15:12:52.972062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 501/1276] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:12:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:12:52.978913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:52.980402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:52.989690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.990383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:52.991638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:52.992299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:53.023225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b52ff7ad-f681-4f7c-b127-d16860d9e5a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049376 Jul 20 15:12:53.023769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 502/1277] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:53.210228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:53.210877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:53.218909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:53.219550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:53.220507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:53.221084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.780290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:12:56.782642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:12:56.784364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:12:56.785435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:12:56.786646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:56.788077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:12:56.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) Jul 20 15:12:56.790699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:12:56.794928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7b707d31-6a19-4691-81bd-5d8a885bdcd8) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:12:56.794928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9a3e8dc-0a6c-4789-ae06-c2c6243baadb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 63.639524 Jul 20 15:12:56.794928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 776/1278] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:11:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 63641 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:12:56.794928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:56.795299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:56.798713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.799191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.799949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.800414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.864567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:12:56.873175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aad33548-7ac7-4400-bf9f-5b67a391cb6c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087348 Jul 20 15:12:56.874000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 777/1279] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:12:56 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:56.884524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:56.885513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:56.891586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.892025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.892814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.893238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.919884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:12:56.923552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-890544ae-a0e4-4fc5-8183-5c49c6279d9f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044760 Jul 20 15:12:56.924110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 778/1280] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:56.928695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:56.929593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:56.937120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.937565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.938332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.938757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.968657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b29c0a0-af21-46c0-af86-6b38eb9024ff tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043534 Jul 20 15:12:56.969203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 779/1281] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:12:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:12:56.973562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:56.974447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:56.980151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.980557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:56.981999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:56.982148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:57.015539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-078b51ab-abf2-4bba-9e0a-d8fd103e5fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044803 Jul 20 15:12:57.016409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 780/1282] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:56 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:57.894834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:57.895920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:57.905612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:57.906069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:57.906833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:57.907304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:57.969236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:12:57.976964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5050f218-8429-4022-a3ab-79dfe2d2cd31 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088554 Jul 20 15:12:57.977540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 781/1283] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:12:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:57.981996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:57.982915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:57.988759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:57.989211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:57.989991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:57.990514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:58.019365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a38a00f1-e2bf-4171-9a54-b3e7ec61dfc5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040809 Jul 20 15:12:58.019898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 782/1284] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:57 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:58.024723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:58.025625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:58.031963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:58.032413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:58.033164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:58.033595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:58.063260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e14b668-28e1-4c05-bb6a-67599f79efb3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042014 Jul 20 15:12:58.063806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 783/1285] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:12:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:12:58.068165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:58.069051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:58.074680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:58.075194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:58.075946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:58.076376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:58.104813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e31c1a93-a6e9-4b65-a6f8-ed7d15a28ebe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040059 Jul 20 15:12:58.105342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 784/1286] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:58.996347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:58.997209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:59.003927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.004701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.005437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.005872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.067776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:12:59.075966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ff1502c-f6d2-4818-a626-63b4668827f7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.085015 Jul 20 15:12:59.076537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 785/1287] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:12:58 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:12:59.081142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:59.082059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:59.088452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.088874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.089644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.090105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.121119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f87025be-cd89-49af-9151-4a7c0a232841 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043546 Jul 20 15:12:59.121714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 786/1288] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:12:59.126336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:59.127277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:59.133189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.133640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.134379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.134808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.164548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebf78d0f-6a20-4e07-8b37-a714a92dc171 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041822 Jul 20 15:12:59.165090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 787/1289] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:12:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:12:59.169494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:12:59.170374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:12:59.176148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.176598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.177364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:12:59.177798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:12:59.206723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-800b4e6d-3935-44bd-83c6-cec2d857df84 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040584 Jul 20 15:12:59.207221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 788/1290] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:12:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:00.097253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:00.098333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:00.107740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.108192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.108953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.109449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.174949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:00.183433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-939ddc33-4e69-4a8b-968b-0d5b6e3bde78 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092301 Jul 20 15:13:00.184299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 789/1291] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:00 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:00.188921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:00.189833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:00.196046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.196483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.197230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.197846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.227395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e10f61d-b21d-40dc-908b-6b04d7a230b1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041996 Jul 20 15:13:00.227976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 790/1292] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:00.234458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:00.235404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:00.241601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.242050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.242807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.243304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.273620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63aa7fc5-b503-4eb9-b023-deac19cd4520 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044591 Jul 20 15:13:00.274184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 791/1293] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:00.278745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:00.279671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:00.285688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.286136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.286898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:00.287374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:00.313268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:00.318255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0892d434-d442-49a1-b393-3ecd3f6dfc6b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043132 Jul 20 15:13:00.318813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 792/1294] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:01.204942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:01.205927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:01.217247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.217693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.218457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.218890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.293886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:01.301648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9d9958ab-d9e0-4f62-9b25-d8df1964a247 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.103464 Jul 20 15:13:01.302184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 793/1295] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:01 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:01.306662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:01.307599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:01.313273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.313726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.314488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.314929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.344238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7dded9f8-1382-4ffb-b576-56b0069b5ec7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041050 Jul 20 15:13:01.344817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 794/1296] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:01 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:01.349239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:01.350127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:01.356011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.356586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.357380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.357814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.386409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3fc3d431-598c-48d4-a02b-adc6f5d695c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040620 Jul 20 15:13:01.386954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 795/1297] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:01.391472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:01.392371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:01.399732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.400175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.400933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:01.401366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:01.429842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-522c8267-0bd8-4a5c-9ade-eb222d749b17 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041846 Jul 20 15:13:01.430390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 796/1298] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:02.322708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:02.323852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:02.331518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.331968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.332732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.333168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.395395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:02.406005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-523d76ea-9c79-4c26-a087-d2a37fe5a2af tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089752 Jul 20 15:13:02.406568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 797/1299] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:02 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:02.411083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:02.412024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:02.417880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.418325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.419080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.419524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.447820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b5c643d1-1b79-4a69-9e48-418d15fe7224 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040266 Jul 20 15:13:02.448393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 798/1300] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:02 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:02.452767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:02.453667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:02.459345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.459781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.460514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.460932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.492257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-60d937e7-3776-4c8b-ae04-e229b968f723 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042794 Jul 20 15:13:02.492925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 799/1301] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:02.497694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:02.498611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:02.504922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.505357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.506179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:02.506638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:02.538543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7cefc897-928e-4fab-88dd-864b8bd4d07b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044162 Jul 20 15:13:02.539371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 800/1302] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:03.215588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=18 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:03.422032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:03.422625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:03.429537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.429986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.430745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.431218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.494471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:03.502642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44140cfb-335d-40e8-abc0-0143142918fc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.084560 Jul 20 15:13:03.503782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 801/1303] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:03 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:03.508206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:03.509103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:03.515205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.515654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.516411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.516841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.546025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-667d0162-bfb2-4cf0-8700-454e4d712164 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041683 Jul 20 15:13:03.546776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 802/1304] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:03.551095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:03.551977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:03.557874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.558347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.559080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.559588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.588525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f7035e60-d280-4d21-b446-ffc5c94eebe3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040979 Jul 20 15:13:03.589044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 803/1305] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:03.593465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:03.594373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:03.600035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.600478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.601224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:03.601656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:03.630811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-801b3375-6803-490c-8018-5f6422387739 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040766 Jul 20 15:13:03.631388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 804/1306] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:04.476098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:13:04.476648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:13:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:13:04.520270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:04.520863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:04.527949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.528389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.529142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.529546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.632707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:04.640656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3f60382c-f815-4d98-a544-22b43d90f886 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.124202 Jul 20 15:13:04.641230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 805/1307] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:04 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:04.650179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:04.650179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:04.652869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.653478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.654246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.654678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.683436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-77a51652-9924-447c-a4fa-deed4003f6f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041132 Jul 20 15:13:04.684005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 806/1308] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:04.691198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:04.692145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:04.698109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.698698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.699480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.699916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.728981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-23a5b922-89e1-44ba-842f-9b6e26025ec6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043900 Jul 20 15:13:04.729561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 807/1309] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:04.734145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:04.735079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:04.740898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.741349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.742112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:04.742541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:04.771829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1058bfc-ffe3-4cbe-a0d1-5fe0100cc7ab tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041213 Jul 20 15:13:04.772672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 808/1310] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:05.665823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:05.667381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:05.682447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.683736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.685298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.685962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.711367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:05.762480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:05.770760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-419359c5-30fc-45b9-b695-d82fa909c978 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.115434 Jul 20 15:13:05.771356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 809/1311] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:05 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:05.775970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:05.778031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:05.782628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.783105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.783879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.784305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.814577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3780dd0b-88a3-40b2-9594-d444190a4f23 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042224 Jul 20 15:13:05.815152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 810/1312] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:05.819588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:05.820471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:05.826490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.826934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.827686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.828106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.856521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-10edef2c-90e7-4e92-8c93-e7c7b5e9fb6e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040299 Jul 20 15:13:05.857047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 811/1313] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:05.861362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:05.862237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:05.868336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.868774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.869514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:05.869939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:05.904313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-095c4acf-c1f9-4cdf-8e73-31cdb5b81e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046252 Jul 20 15:13:05.904875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 812/1314] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:06.791370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:06.792075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:06.799532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.799981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.800741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.801196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.863750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:06.871848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dad7efa1-43ea-4d85-a536-8fcdbfa9064a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086501 Jul 20 15:13:06.872418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 813/1315] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:06 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:06.876962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:06.877879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:06.884265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.884738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.885500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.885971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.915262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e19ac40b-a6a2-4ae4-b93f-9443e4d59ffd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041810 Jul 20 15:13:06.915818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 814/1316] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:06.920279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:06.921166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:06.924315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=56 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:06.927432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.927878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.928952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.929056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.958908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6331380b-dd53-44c5-b08b-390c58a580ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042060 Jul 20 15:13:06.959503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 815/1317] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:06.963913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:06.964808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:06.970744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.971226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:06.971983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:06.972417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:07.000955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8fa02dc4-a7b0-4f01-b639-9fa9811efdb6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040454 Jul 20 15:13:07.001496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 816/1318] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:06 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:07.889538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:07.890291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:07.897189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:07.897643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:07.898405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:07.898842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:07.961319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:07.969185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44bf5396-06b0-420c-9c89-f123c09b52d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.083561 Jul 20 15:13:07.969752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 817/1319] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:07 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:07.974279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:07.975194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:07.981235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:07.981710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:07.982471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:07.982909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.012678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-585adc6c-c24f-4f60-86d3-87a22cb22194 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041931 Jul 20 15:13:08.013227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 818/1320] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:07 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:08.017623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:08.018536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:08.024257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.024706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.025454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.025888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.054615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f6006d3-28cd-4f95-8260-1caf78497476 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040424 Jul 20 15:13:08.055142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 819/1321] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:08 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:08.059872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:08.060782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:08.066751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.067232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.067985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.068422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.098315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-508e0ad6-7757-441f-ba51-d2b1c2c5055d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041903 Jul 20 15:13:08.098852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 820/1322] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:08.224068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:13:08.224919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:13:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:13:08.986396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:08.987000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:08.993835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.994315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:08.995085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:08.995511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.060819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:09.068856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-79ce2715-9c38-4487-8249-d45775ae7122 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086275 Jul 20 15:13:09.069411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 821/1323] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:08 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:13:09.073999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:09.074998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:09.080925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.081351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.082101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.082534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.111065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-78f88a73-8018-43fe-88e0-e71266c934a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040651 Jul 20 15:13:09.111640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 822/1324] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:09 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:09.116108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:09.116992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:09.122925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.123398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.124134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.124559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.153152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-00d472e2-1a6e-4e19-b431-b1d35d447a5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040438 Jul 20 15:13:09.153690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 823/1325] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:09.157988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:09.158953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:09.164895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.165331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.166065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:09.166486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:09.194924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9d8bc499-63e2-401e-8a53-70c517c016a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040209 Jul 20 15:13:09.195425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 824/1326] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:10.090075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:10.090972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:10.097738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.098179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.098934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.099392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.170529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:10.178303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-323bbb64-8e7a-4c9c-93ef-20a9470685c2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095002 Jul 20 15:13:10.178861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 825/1327] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:10 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:10.183385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:10.184306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:10.186923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.190360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.190805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.191587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.192017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.196142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:10.220909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74d1b3ac-8bda-4f8b-be14-2867b26c25e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041036 Jul 20 15:13:10.221494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 826/1328] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:10 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:10.225909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:10.226795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:10.232334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.232775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.233520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.233944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.262755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad32902a-c5d6-4c04-b856-b0c50fa11ef9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040280 Jul 20 15:13:10.263337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 827/1329] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:10.267680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:10.268567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:10.274019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.274463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.275236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:10.275663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:10.305185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9e976db-eb7e-4f8f-9d7b-b46810bf94f3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040888 Jul 20 15:13:10.305736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 828/1330] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:11.199618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:11.200422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:11.207254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.207704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.208458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.208887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.272771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:11.280776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fb5e3a3c-0e98-4e86-95cc-2bdc8cafdd64 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087724 Jul 20 15:13:11.281333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 829/1331] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:11 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:11.285776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:11.286683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:11.292393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.292831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.293683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.294115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.322790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3cc72b3a-c224-475d-976c-c417daf42545 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040469 Jul 20 15:13:11.323368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 830/1332] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:11 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:11.327881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:11.328765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:11.334417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.334856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.335629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.336058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.366486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1be5874c-a183-415d-a2f2-e94d4fca715b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042162 Jul 20 15:13:11.367027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 831/1333] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:11.371366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:11.372297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:11.377908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.378352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.379375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:11.379855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:11.408282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-09a5e2e7-86de-44ff-8853-547964d487b5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040274 Jul 20 15:13:11.408813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 832/1334] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:12.301543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:12.302682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:12.311998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.312444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.313204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.313633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.385077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:12.392832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1987377e-56d0-4724-80e7-ba61506c35bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097817 Jul 20 15:13:12.393403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 833/1335] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:12 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:12.397830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:12.398732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:12.404469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.404910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.405665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.406091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.434781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-481e393a-f949-4162-8261-bc605765a390 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040416 Jul 20 15:13:12.435360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 834/1336] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:12 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:12.439765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:12.440642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:12.448248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.448248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.448248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.448248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.484114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a1dac6c5-49be-4666-9c3a-6fe112ee17b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047526 Jul 20 15:13:12.484837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 835/1337] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:12.489557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:12.490478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:12.500258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.500765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.501543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:12.501968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:12.533642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-379c4835-dbbc-4dc4-94ec-5f52a4d2356b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047852 Jul 20 15:13:12.534194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 836/1338] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:13.410200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:13.410918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:13.417876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.418325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.419357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.419794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.491981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:13.499728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3fc23fca-02bb-4b3a-9e9d-0372ff6bc786 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093817 Jul 20 15:13:13.500289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 837/1339] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:13 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:13.505110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:13.506007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:13.511659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.512134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.512897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.513330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.541187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c8ee4721-144f-4c05-965d-6c727a20f0a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039904 Jul 20 15:13:13.541735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 838/1340] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:13.546224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:13.547107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:13.553084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.553531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.554280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.554713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.584211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0dc2e019-fbc0-4330-b039-4e563b64a7c7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041501 Jul 20 15:13:13.584759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 839/1341] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:13.589225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:13.590136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:13.595845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.596291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.597037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:13.597471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:13.625972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-888271ae-6f05-442b-92e5-632eb9a78879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040274 Jul 20 15:13:13.626536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 840/1342] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:14.517249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:14.517928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:14.524847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.525306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.526055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.526561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.589570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:14.597533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7ec3ee25-759e-4325-98dd-14cc0a548fea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.084260 Jul 20 15:13:14.598100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 841/1343] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:14 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:14.602541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:14.603476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:14.609141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.609590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.610339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.610773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.636866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:14.641799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ad9a749-7518-4b5e-a915-d51eabf5d9fc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042758 Jul 20 15:13:14.642366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 842/1344] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:14.647079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:14.647979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:14.653811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.654261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.655031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.655469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.685501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-de208675-feaf-4f32-8f81-7754d7b794f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042169 Jul 20 15:13:14.686030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 843/1345] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:14.690353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:14.691275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:14.696792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.697228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.698086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:14.698502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:14.726689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e8c58f36-50b0-49f7-8ca6-e6be73d7454e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039721 Jul 20 15:13:14.727281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 844/1346] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:15.617989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:15.619094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:15.631037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.631817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.633043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.633782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.696874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:15.705091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72dd61d3-2453-4362-811b-ef577ac9a2e7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093596 Jul 20 15:13:15.705662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 845/1347] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:15 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:13:15.714904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:15.716632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:15.726251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.726708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.727913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.728365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.758849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3f8bd22-95b8-4904-a8b5-2e56e866d13e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052174 Jul 20 15:13:15.759449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 846/1348] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:15.764978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:15.766671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:15.773470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.773993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.774756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.775331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.805236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-64d776d5-59ed-4beb-ae42-db6ab3d2839a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044734 Jul 20 15:13:15.805795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 847/1349] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:15.810503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:15.811461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:15.817419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.817866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.818653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:15.819079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:15.847914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ffc137a4-a3d6-43cb-9271-719649d6e01d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041130 Jul 20 15:13:15.848453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 848/1350] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:15 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:16.726111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:16.727229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:16.738379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.739017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.740398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.741115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.805714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:16.813368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7099dee2-a5f3-4711-99ca-ab682d1d40dc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093610 Jul 20 15:13:16.813944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 849/1351] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:16 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:16.818383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:16.819604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:16.827362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.827807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.828563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.828990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.857498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b7123e7-aabd-4ff1-b2a9-8f28b3cc2d0c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042600 Jul 20 15:13:16.858048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 850/1352] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:16.862503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:16.863421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:16.869113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.869582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.870298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.870724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.900549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b5e40d32-7f9b-44f7-8b12-fc79eb3592a3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041495 Jul 20 15:13:16.901095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 851/1353] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:16.905493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:16.906371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:16.912059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.912476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.913246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:16.913672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:16.941953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7ef82ee-749b-494e-a894-527931839f6e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039922 Jul 20 15:13:16.942481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 852/1354] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:17.834065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:17.835151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:17.846205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.846951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:17.848259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.848976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:17.916777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:17.925112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ece1d5ac-60a6-4988-897d-739219c98856 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097630 Jul 20 15:13:17.925672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 853/1355] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:17 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:17.930423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:17.931324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:17.937270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.937712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:17.938459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.938883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:17.968787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b264721c-cafd-4a03-bdb2-21122beb728c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042111 Jul 20 15:13:17.969328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 854/1356] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:17 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:17.973764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:17.974643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:17.980258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.980697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:17.981475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:17.981897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:18.010171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c458458f-b261-4227-b8a0-fb83e7096da5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039891 Jul 20 15:13:18.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 855/1357] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:18.014992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:18.015890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:18.021603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:18.022036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:18.023144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:18.023623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:18.051709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4589141-83e4-4b49-8bc5-2d436d2d7cd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040053 Jul 20 15:13:18.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 856/1358] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:18.946827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:18.947950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:18.958650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:18.959474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:18.960737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:18.961504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.026009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:19.034412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f9e36d21-540b-4ce8-b55a-3115a0725dc8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094585 Jul 20 15:13:19.035000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 857/1359] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:18 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:19.039580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:19.040491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:19.046690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.047186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.047940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.048364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.080336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6dd06021-1fc1-48ea-8c57-ed3b2e97ab02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044250 Jul 20 15:13:19.080892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 858/1360] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:19.085390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:19.086298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:19.092345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.092787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.093541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.093965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.117461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:19.132353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5107ab6a-d5b9-4b42-bc6b-a28e36748be4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050510 Jul 20 15:13:19.132911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 859/1361] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:19.137380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:19.138285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:19.144350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.144807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.145607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:19.146046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:19.175687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4b3b6734-4b06-4c39-ad45-439f8ce0f3f9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041751 Jul 20 15:13:19.176234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 860/1362] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:19 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:20.055795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:20.056891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:20.068221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.069058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.070314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.071031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.135641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:20.144214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e8965d2b-782c-4bd5-b7d7-3cb4b888258a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095028 Jul 20 15:13:20.144788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 861/1363] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:20 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:20.149306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:20.150217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:20.156507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.156975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.157797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.158233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.187290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f6bddc40-d878-40b1-a4ed-85e3bd7be642 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041440 Jul 20 15:13:20.187836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 862/1364] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:20 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:20.192290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:20.193188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:20.198990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.199471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.200312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.200775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.230783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-601181c3-05f5-41c7-87d5-6f72d4be852f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041950 Jul 20 15:13:20.231428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 863/1365] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:20.235779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:20.236687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:20.242542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.243011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.243813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:20.244226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:20.273487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9abbf74-21d6-46c7-ae06-bb8431115e13 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041130 Jul 20 15:13:20.274030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 864/1366] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:20.897221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:13:20.897800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:13:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:13:21.165608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:21.166625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:21.177239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.177972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.179268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.179971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.243794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:21.251546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-427dd71e-aa41-4c06-81ba-1bafbb04ea2c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092714 Jul 20 15:13:21.252098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 865/1367] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:21 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:21.256784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:21.257687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:21.263515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.263962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.264706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.265127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.294543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87fe331c-2ee7-4997-a37e-9f98a6f29eee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041478 Jul 20 15:13:21.295080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 866/1368] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:21.299423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:21.300294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:21.306247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.306786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.307560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.307981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.336406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d5575a40-e338-4511-bc05-3ea3f4e41b86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040256 Jul 20 15:13:21.336957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 867/1369] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:21.341380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:21.342261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:21.348101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.348533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.349294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:21.349714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:21.379848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba9286a3-4a9c-4a4c-b729-30a0ac71d2f6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041939 Jul 20 15:13:21.380382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 868/1370] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:21 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:22.270132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:22.270839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:22.278321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.278788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.279585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.280017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.351446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:22.359708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f875a08f-1a01-4c52-9cbe-2097e310d408 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093524 Jul 20 15:13:22.360276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 869/1371] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:22 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:22.364891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:22.365923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:22.372009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.372455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.373237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.373825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.403832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9a69bbe-3e70-4e4a-870e-e6595992bc27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042537 Jul 20 15:13:22.404384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 870/1372] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:22 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:22.408902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:22.409850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:22.415922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.416370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.417272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.417614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.446580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-09ea6462-a315-4c9d-96e2-b544db97563a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041209 Jul 20 15:13:22.447125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 871/1373] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:22.451602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:22.452490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:22.458705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.459278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.460218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:22.460650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:22.492265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3ce6e45c-a792-41e5-8144-588a9cd9a956 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044051 Jul 20 15:13:22.492854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 872/1374] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:23.381378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:23.382564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:23.391122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.391573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.392329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.392750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.456079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:23.463932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c4d47d0-5982-4997-b53f-4cf47b1ca084 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089543 Jul 20 15:13:23.464490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 873/1375] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:23 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:23.469204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:23.470113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:23.475944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.476446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.477209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.477638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.507387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f2b2c28-4b37-4ec5-bffe-c18c839634db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041702 Jul 20 15:13:23.507938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 874/1376] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:23.512497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:23.513395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:23.519090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.519533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.520276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.520703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.548852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30129136-badf-43de-80dc-56f3d48cc394 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039889 Jul 20 15:13:23.549392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 875/1377] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:23.554011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:23.554897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:23.557620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.562634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.562634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.563013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:23.563234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:23.568095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:23.593161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-113e0bef-f767-485e-a618-1c37985a7e7b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042794 Jul 20 15:13:23.593729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 876/1378] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:24.485683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:24.486724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:24.497619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.498067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.498832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.499302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.565475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:24.573219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3761784f-0f3b-4bcc-92f2-40870708f42a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094196 Jul 20 15:13:24.573777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 877/1379] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:24 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:24.578306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:24.579250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:24.585252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.585692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.586552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.586973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.616483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbcd1022-825d-4be4-81f2-5e1a0e591f9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041711 Jul 20 15:13:24.617025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 878/1380] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:24.621633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:24.622510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:24.628077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.628542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.629568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.629994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.659016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6be88828-b69a-470c-ab97-109b951d3355 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040944 Jul 20 15:13:24.659579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 879/1381] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:24.663908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:24.664889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:24.670571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.671007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.671784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:24.672211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:24.700985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e46c9f3-0f73-45a0-b59a-f304234c3a1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040479 Jul 20 15:13:24.701520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 880/1382] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:25.596579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:25.597595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:25.607447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.607877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.608617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.609029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.686206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:25.698040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-edbb42a7-a26e-4e1a-bdce-8f1ca62af7a1 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.107481 Jul 20 15:13:25.698723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 881/1383] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:25 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:25.703203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:25.704139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:25.711078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.711556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.712315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.712794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.742984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4f52d471-6660-4832-9c19-dd43e9093bde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043136 Jul 20 15:13:25.743466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 882/1384] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:25.748103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:25.748986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:25.754853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.755366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.756109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.756537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.788592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1ee7051-4753-488d-ba7f-7ad09f008081 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044115 Jul 20 15:13:25.789347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 883/1385] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:25.793816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:25.794709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:25.800600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.801038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.801819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:25.802249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:25.831630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc1d705e-7ab4-4e05-9ac9-12c0e5e2faf2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041303 Jul 20 15:13:25.832174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 884/1386] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:26.718778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:26.719653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:26.728479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.729094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.730131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.730802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.800384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:26.812137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d1df26b-273d-4132-9ece-4c314be9e5a3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098957 Jul 20 15:13:26.812854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 885/1387] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:26 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:26.817335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:26.818240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:26.825505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.825946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.826694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.827117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.856563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2188226-a38e-4c6d-8d9e-58f07c9a8075 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042707 Jul 20 15:13:26.857113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 886/1388] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:26.861572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:26.862461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:26.868271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.868736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.869484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.869911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.900181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3a119741-a340-4a13-a23e-cc894ab0f91a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042084 Jul 20 15:13:26.900716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 887/1389] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:26.905085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:26.905960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:26.911567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.912007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.912774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:26.913203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:26.941323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-12fe3c10-b887-45e7-8987-bb7fddefe602 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039664 Jul 20 15:13:26.941859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 888/1390] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:27.833453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:27.834548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:27.845328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.846041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:27.847365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.848078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:27.913280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:27.921561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb623c88-e2a5-4ab8-be1c-9a7bdec00953 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094573 Jul 20 15:13:27.922130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 889/1391] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:27 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:27.926590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:27.927557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:27.933436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.934033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:27.934791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.935266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:27.964852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6596dbe9-adb0-4568-a1d1-24740f2837e3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041714 Jul 20 15:13:27.965457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 890/1392] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:27 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:27.969873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:27.970765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:27.976691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.977140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:27.977886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:27.978326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:28.009245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-344a6b0e-99a6-492a-84f3-c100bd5bbc85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.042798 Jul 20 15:13:28.009803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 891/1393] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:28.014154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:28.015036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:28.020899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:28.021343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:28.022095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:28.022526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:28.046235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:28.050903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-866910ee-d9ef-4f3a-95a5-25bc5852d560 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040182 Jul 20 15:13:28.051476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 892/1394] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:28.758482 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:13:28.759363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:13:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:13:28.942999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:28.944139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:28.954900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:28.955688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:28.957178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:28.957570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.023871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:29.031548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1cba9c09-332b-4765-98eb-0bbf059b1857 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095360 Jul 20 15:13:29.032091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 893/1395] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:28 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:29.036517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:29.037399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:29.042928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.043437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.044179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.044603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.074102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c1b52e64-b8fa-4d14-b826-28c70cb9b28d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041021 Jul 20 15:13:29.074692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 894/1396] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:29.079106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:29.080036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:29.087475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.088089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.089227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.089913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.123932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bfbe325e-dd7e-47a9-84ce-89b5697ef1ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048232 Jul 20 15:13:29.124481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 895/1397] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:29.129121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:29.130002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:29.135423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.135862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.136600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:29.137074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:29.165458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-173d5377-6e33-4d75-a4a9-5242e043db00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040021 Jul 20 15:13:29.165993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 896/1398] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:30.053359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:30.054431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:30.064999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.065736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.066967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.067720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.131768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:30.139871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-920c3f12-14e7-456e-9b87-7061edf45556 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093477 Jul 20 15:13:30.140424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 897/1399] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:30 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:30.144794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:30.145692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:30.151305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.151762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.152862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.153292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.183406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c1923124-58e9-4ce1-9eca-b9815e77777a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041971 Jul 20 15:13:30.183995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 898/1400] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:30 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:30.188381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:30.189257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:30.194879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.195464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.196202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.196620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.225370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0cef160d-f7c1-48ec-97cc-d232127263db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040367 Jul 20 15:13:30.225878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 899/1401] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:30.230270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:30.231190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:30.237050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.237502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.238256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:30.238690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:30.267275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c9cf2ab7-1564-4b38-9053-2e952c396413 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040354 Jul 20 15:13:30.267814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 900/1402] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:31.157035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:31.157718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:31.164578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.165050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.165796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.166223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.241010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:31.249308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36fde9ef-db7c-44c2-9239-7770b4fb5943 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096152 Jul 20 15:13:31.249869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 901/1403] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:31 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:31.254633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:31.255560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:31.261685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.262129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.262878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.263338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.292630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d9211c43-b96b-4159-850d-16ca6ed56551 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041799 Jul 20 15:13:31.293214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 902/1404] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:31.297578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:31.298463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:31.304608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.305103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.305845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.306267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.340159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc19250e-4336-44f4-82ba-a350b8026ed8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045573 Jul 20 15:13:31.340860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 903/1405] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:31.345691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:31.347034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:31.355244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.355858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.356608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:31.357040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:31.385500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-13571e11-e264-42c5-b1fb-e9b6f7979950 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043710 Jul 20 15:13:31.386039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 904/1406] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:32.270931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:32.272053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:32.283268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.284022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.285301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.286070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.350046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:32.357923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c6f1ceb-e379-4f29-a660-e793e42a0503 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093512 Jul 20 15:13:32.358495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 905/1407] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:32 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:32.362990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:32.363937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:32.369698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.370147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.370905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.371368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.400401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3517e787-f1c2-4081-bb93-c4c6218d9002 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040911 Jul 20 15:13:32.400943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 906/1408] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:32.405402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:32.406286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:32.411979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.412419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.413166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.413602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.443023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8f6e2942-da49-4783-9f9b-63381b6beda2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041117 Jul 20 15:13:32.443597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 907/1409] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:32.448014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:32.448905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:32.454546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.454993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.455780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:32.456210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:32.486860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-01891345-4942-4038-9c22-4d4c1ef04659 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041873 Jul 20 15:13:32.487871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 908/1410] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:33.379963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:33.381031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:33.390083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.390535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.391328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.391764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.410061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:33.462512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:33.470148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-38d44fb2-cf90-4c00-b9fd-988f6aa8992a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097239 Jul 20 15:13:33.470710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 909/1411] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:33 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:33.475142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:33.476063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:33.484559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.484559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.484559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.484559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.513358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1c145658-9df3-410a-a1f5-dbb0969f1867 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041597 Jul 20 15:13:33.513906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 910/1412] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:33 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:33.518370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:33.519318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:33.525006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.525451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.526204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.526638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.555358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-93172356-9fe8-4e68-8dfb-d995ffcbde75 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040450 Jul 20 15:13:33.555897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 911/1413] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:33.560323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:33.561206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:33.567129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.567579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.568327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:33.568755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:33.597339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87e456c3-d492-4bb8-9754-dda0b7d78892 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040505 Jul 20 15:13:33.597878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 912/1414] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:34.495840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:34.497904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:34.504542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.504987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.505738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.506170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.583529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:34.597789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4159c4be-be5d-4f1f-82c3-22bb648f52b3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.108516 Jul 20 15:13:34.598372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 913/1415] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:34 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:13:34.602895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:34.603838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:34.611687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.612304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.613365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.613935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.644895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-26a7a60b-3057-4086-8850-d0bfb752142e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045538 Jul 20 15:13:34.645433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 914/1416] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:34.649990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:34.650861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:34.656573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.657006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.657739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.658167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.686391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b1bf1f3-e4bc-4932-a403-621d674fb7a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040013 Jul 20 15:13:34.686926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 915/1417] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:34.691284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:34.692156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:34.698199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.698636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.699482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:34.699908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:34.729211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9354b23b-383c-4284-b3ee-2294f8239267 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041280 Jul 20 15:13:34.729744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 916/1418] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:35.620040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:35.621046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:35.632424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.633168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.634367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.635125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.709469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:35.717263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1db63360-480f-4b64-be77-cac4c3c4dd54 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.103712 Jul 20 15:13:35.717818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 917/1419] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:35 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:35.722282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:35.723189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:35.729031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.729481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.730272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.730741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.759910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfac0ca6-1662-4276-b814-f16453c76492 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041060 Jul 20 15:13:35.760382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 918/1420] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:35.764876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:35.765815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:35.771460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.771905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.772653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.773059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.801484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4964a871-dd3a-46b1-bf8a-88357c9e44dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040094 Jul 20 15:13:35.802032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 919/1421] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:35.806395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:35.807321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:35.813153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.813602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.814351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:35.814786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:35.843114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-892189cf-3af9-4d90-93dc-1c601686021a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040125 Jul 20 15:13:35.843690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 920/1422] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:36.738045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:36.739245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:36.748800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.749177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.749940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.750370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.815216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:36.823363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-862727cb-71b2-4379-91b5-1b3c944ee74b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091628 Jul 20 15:13:36.823931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 921/1423] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:36 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:36.828461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:36.829323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:36.835067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.835523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.836272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.836702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.865402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cde9322c-44dd-4869-80c0-23f5d339e3d9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040473 Jul 20 15:13:36.865947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 922/1424] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:36 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:36.870329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:36.871356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:36.876978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.877425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.878170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.878889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.907908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30df859a-aa4a-44b6-a359-3d53e855d847 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040995 Jul 20 15:13:36.908438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 923/1425] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:36.913079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:36.913962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:36.920124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.920566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.921303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:36.921733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:36.950466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-573b3557-9ee8-46d9-8f8e-b9bfa558df48 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041083 Jul 20 15:13:36.950994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 924/1426] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:37.844427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:37.845482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:37.856145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.856593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:37.857358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.857792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:37.919201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:37.926907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5f0be6eb-d49a-46bd-8228-b177d60757db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089039 Jul 20 15:13:37.927532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 925/1427] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:37 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:37.932310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:37.933215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:37.935960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.939717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.940167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:37.940928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.941360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:37.945695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:37.972642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-11375c05-e681-42b7-9b1b-12eb3eb497bd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044110 Jul 20 15:13:37.973189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 926/1428] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:37 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:37.977627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:37.978547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:37.984454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.984901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:37.985655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:37.986086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:38.016033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6fd0cae3-00e4-4fe7-95d8-933ec82a3270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041770 Jul 20 15:13:38.016613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 927/1429] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:38.021168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:38.022074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:38.028567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:38.029006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:38.029765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:38.030196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:38.062938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-707d9287-a81a-4ce0-bf55-e8592dd48dbb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045305 Jul 20 15:13:38.063643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 928/1430] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:38.947939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:38.948987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:38.960282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:38.961026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:38.962647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:38.963781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.026506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:39.034678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bdd5f9ef-a49d-4a83-93ac-7cb5800705d8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093230 Jul 20 15:13:39.035279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 929/1431] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:38 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:39.039801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:39.040750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:39.046510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.046965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.047785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.048225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.077950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e86f176-21d9-42a1-9fbd-47f4473131a8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041655 Jul 20 15:13:39.078499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 930/1432] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:39.083089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:39.084018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:39.089944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.090388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.091176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.091609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.119971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fbf1d4da-5f13-4169-a1e7-706fa04be357 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040494 Jul 20 15:13:39.120524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 931/1433] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:39.124969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:39.125879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:39.131685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.132122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.132870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:39.133329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:39.161485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-75b11870-0031-4958-8725-b69f873490e0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039956 Jul 20 15:13:39.162012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 932/1434] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:40.056276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:40.057383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:40.068421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.069048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.070304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.071030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.146564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:40.158245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0abdd4e0-3955-4c8c-a741-40b7e363da70 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.108544 Jul 20 15:13:40.159053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 933/1435] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:40 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:40.163580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:40.164490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:40.170461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.171295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.172051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.172478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.201787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d5fc8966-8c2d-4054-8a88-d8e60462b222 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041698 Jul 20 15:13:40.202343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 934/1436] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:40 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:40.206774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:40.207684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:40.213479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.213912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.214663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.215091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.244005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-97b427fe-a7f6-487a-9908-a6ee0de5e407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040694 Jul 20 15:13:40.244546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 935/1437] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:40.248925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:40.249792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:40.256202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.256649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.257396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:40.257817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:40.286797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7fc32f62-bcac-4bc5-8d15-addb74759416 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041302 Jul 20 15:13:40.287383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 936/1438] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:40 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:41.179609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:41.180688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:41.188665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.189112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.189867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.190292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.252363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:41.260355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-54e39871-b04d-4466-b310-ad1b2f320dde tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087178 Jul 20 15:13:41.260917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 937/1439] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:41 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:41.265410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:41.266630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:41.272727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.273154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.273919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.274345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.302861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d72a795e-acb7-4f86-8a46-b6a1e5c2bce5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040932 Jul 20 15:13:41.303435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 938/1440] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:41 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:41.308594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:41.309465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:41.315262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.315708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.316448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.316878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.346205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bee0690a-9342-4dc9-8d81-9e7d2ca94cd1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041817 Jul 20 15:13:41.346770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 939/1441] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:41.351081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:41.351955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:41.357766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.358206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.358952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:41.359417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:41.389227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7cc52d67-e4ed-417f-afbf-080ecfebfbd5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041495 Jul 20 15:13:41.389779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 940/1442] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:42.281596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:42.282773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:42.293352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.293799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.294585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.295016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.355870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:42.363583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb89c6de-b7fd-476a-ba24-4b9b1bfbebac tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088574 Jul 20 15:13:42.364136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 941/1443] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:42 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:42.368652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:42.369569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:42.375334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.375779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.376574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.376957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.401605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:42.407641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d2b2394b-76d4-4cdb-ad9c-f444aa99609a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042109 Jul 20 15:13:42.408433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 942/1444] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:42.416005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:42.417345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:42.424407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.424845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.425593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.425988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.456899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-698d3031-99a0-40f9-99e5-3cd7acd689c1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047260 Jul 20 15:13:42.457475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 943/1445] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:42.462063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:42.462963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:42.469315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.469579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.470252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:42.470679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:42.501503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-00986f11-8b2a-4405-bfff-9076ae6ce797 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043023 Jul 20 15:13:42.502067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 944/1446] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:43.380885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:43.381544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:43.388293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.388743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.389503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.389936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.452118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:43.460213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb4d6f40-22c8-48c4-ba3b-abcd5fa98fef tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.083610 Jul 20 15:13:43.460762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 945/1447] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:43 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:13:43.465423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:43.466371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:43.472210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.472659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.473413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.473866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.503064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfd9d6e2-eee7-45c3-97bd-83cc9e3657e9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041236 Jul 20 15:13:43.504006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 946/1448] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:43.508508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:43.509413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:43.515245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.515690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.516445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.516856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.545538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81e659ad-e12f-4513-8197-da99809adb09 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040535 Jul 20 15:13:43.546084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 947/1449] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:43.550488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:43.551387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:43.557050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.557498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.558244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:43.558795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:43.587317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80766981-5d01-454d-8dd6-fbd249bc25c5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040247 Jul 20 15:13:43.587861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 948/1450] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:44.481168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:44.481860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:44.489018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.489467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.490218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.490707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.562310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:44.570309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e146ac47-0720-4367-bb26-11fef6761410 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095242 Jul 20 15:13:44.570875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 949/1451] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:44 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:44.575468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:44.576376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:44.582230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.582676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.583487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.583918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.612975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d18f588a-82d2-4377-90cc-c3f1fb964b59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041075 Jul 20 15:13:44.613517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 950/1452] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:44 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:44.618545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:44.619460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:44.625229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.625670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.626415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.626847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.656269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c84c59a1-b201-4d06-9df7-cd6aa3a89da2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041784 Jul 20 15:13:44.656819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 951/1453] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:44.661361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:44.662268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:44.668136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.668582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.669328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:44.669762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:44.698137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2a9b503d-cd56-4168-8387-247fd0773dbe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040325 Jul 20 15:13:44.698678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 952/1454] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:45.590847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:45.591565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:45.599180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.599681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.600524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.601007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.670845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:45.678583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a41b04c2-b26c-4b4f-b706-ef3bdc0099bd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094261 Jul 20 15:13:45.679180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 953/1455] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:45 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:45.683786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:45.684704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:45.690660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.691116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.691880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.692315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.722087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9dbbd8f-fb1d-4219-b55b-7b9017ddf9dc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041872 Jul 20 15:13:45.722619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 954/1456] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:45.727079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:45.727961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:45.734129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.734578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.735366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.735802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.770262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b29c73d9-fb3a-49c1-b4e4-eedf21892a8b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046471 Jul 20 15:13:45.771000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 955/1457] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:45.775293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:45.776151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:45.782009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.782457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.783262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:45.783687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:45.812641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c759058d-a41c-4887-b86f-506e3d7ceb31 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040861 Jul 20 15:13:45.813549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 956/1458] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:46.699397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:46.700041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:46.707000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.707508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.708269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.708702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.777201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:46.785375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-11702fba-84d0-42de-b8a5-c6e9e4dcfa9e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092500 Jul 20 15:13:46.785956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 957/1459] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:46 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:46.790405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:46.791371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:46.797107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.797558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.798313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.798873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.828153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe0a4185-db3e-4ff1-97d2-d311c61c0054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041186 Jul 20 15:13:46.828704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 958/1460] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:46 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:46.833209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:46.834106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:46.840176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.840619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.841374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.841806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.858023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:46.871387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6166a313-3cf7-42f4-a736-8ac6f1315f57 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.041698 Jul 20 15:13:46.871937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 959/1461] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:46.876289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:46.877176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:46.883065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.883521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.884271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:46.884704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:46.914838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85f7fa81-9b87-481c-8d02-f648ff2e867b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041859 Jul 20 15:13:46.915435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 960/1462] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:47.806769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:47.807864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:47.815023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.815521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.816301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.816762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.879711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:47.891615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2f4faa50-406a-440a-84db-f62cf4f31676 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091273 Jul 20 15:13:47.892452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 961/1463] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:47 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:47.897041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:47.897965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:47.904105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.904545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.905295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.905718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.935492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7df29812-079e-4fcb-b632-f4e1f0040253 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041938 Jul 20 15:13:47.936038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 962/1464] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:47 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:47.940417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:47.941298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:47.947190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.947634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.948371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.948814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.977194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3744388c-f6c4-4a20-8747-a1bd1c791827 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.040208 Jul 20 15:13:47.977729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 963/1465] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:47.982086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:47.982968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:47.988870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.989307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:47.990041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:47.990466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:48.018990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3c6a3402-8e7c-454a-a420-44a0365654e3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040335 Jul 20 15:13:48.019556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 964/1466] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:48.912848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:48.913910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:48.925163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:48.925905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:48.927318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:48.928034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.000176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:49.011777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3790026-43ec-4cc0-a29a-c403dd5dca14 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.105140 Jul 20 15:13:49.012340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 965/1467] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:48 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:49.016781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:49.017693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:49.023555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.023999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.024747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.025172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.052934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6e40910f-8dac-4d3e-be31-88a073160d4a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039623 Jul 20 15:13:49.053472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 966/1468] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:49 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:49.057989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:49.058890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:49.064502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.064934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.065664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.066085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.094326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24d0be71-9f87-4528-917f-1d1ad41a65f4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.039836 Jul 20 15:13:49.094866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 967/1469] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:13:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:49.099157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:49.100026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:49.105792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.106231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.106962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:49.107415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:49.136234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-817673fc-5b1e-4997-8809-28f507e0599a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040374 Jul 20 15:13:49.136834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 968/1470] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:50.033084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:50.034097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:50.044291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.044773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.045577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.046053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.109681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:50.121412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3284a6c6-622e-4580-83b1-b6fae750a7d9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095011 Jul 20 15:13:50.121923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 969/1471] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:50 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:50.126298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:13:50.127628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:50.133811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.134254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.135032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.135522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.236244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2ae3484a-ef5a-4e0b-a7fb-6d289109dd13 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/action" status: 200 len: 41203 microversion: 2.1 time: 0.112485 Jul 20 15:13:50.244012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 970/1472] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:13:50 2022] POST /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92/action => generated 41203 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:13:50.246664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:13:50.247627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:50.255341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.255782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.256497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.256962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.346816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad249182-d8d1-412f-8cfa-ae7d3fd46ea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action" status: 200 len: 37872 microversion: 2.20 time: 0.107343 Jul 20 15:13:50.347891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 971/1473] 10.222.0.15 () {62 vars in 1318 bytes} [Wed Jul 20 15:13:50 2022] POST /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d/action => generated 37872 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:13:50.356314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:50.357866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:50.369815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.370548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.371521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:50.371942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:50.402077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7ca6fab-24a9-409f-87eb-7b9053dc0897 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052583 Jul 20 15:13:50.402617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 972/1474] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:51.138974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:51.139658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:51.146606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:51.147081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:51.147854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:51.148287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:51.213238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:51.226056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c464025-c354-435c-99fc-67e29347a90d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090620 Jul 20 15:13:51.226683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 973/1475] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:51 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:51.415692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:51.416776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:51.423932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:51.424381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:51.425146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:51.425574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:51.605810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ebda7e0-0d57-43d6-bf19-6c1f1d31d93a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.195369 Jul 20 15:13:51.606356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 974/1476] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:13:52.248429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:52.249510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:52.260481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:52.261276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:52.262244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:52.262666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:52.323739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:13:52.335552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b6185036-4495-40bc-8c75-90d23b897809 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.093420 Jul 20 15:13:52.336370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 975/1477] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:13:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:13:52.621170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:52.622107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:52.625205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:52.629570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:52.630019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:52.630776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:52.631240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:52.657333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:13:52.662321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be2d1122-0952-4ca2-8d89-1484cb974235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047995 Jul 20 15:13:52.662870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 976/1478] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:13:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:13:53.274714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:13:53.275759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:13:53.283423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:53.283903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:13:53.284705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:13:53.285158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:03.280777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:03.305063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:14:43.724029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:14:43.725263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:14:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:14:52.037734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:14:52.040063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:14:52.041959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:14:52.043391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:14:52.046544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:52.048201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:14:52.049652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) Jul 20 15:14:52.051193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:14:52.053006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c6594e9-b086-41ee-b4ed-0770ae5a725f) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:14:52.053006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8e27888d-7901-4a78-b75c-7bb97e957137 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 118.836239 Jul 20 15:14:52.053006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 503/1479] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:12:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 118838 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:14:52.053006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:52.053432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:52.054728 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:52.055182 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:52.055941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:52.056372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:52.121583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:14:52.129671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-820ac322-0c31-4417-8f20-71e8e62dda76 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086542 Jul 20 15:14:52.130236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 504/1480] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:14:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:14:52.134888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:52.135993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:52.141771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:52.142235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:52.142985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:52.143456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:52.173620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fbdc91b4-09aa-45fc-891d-65c6b8f0ba57 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042368 Jul 20 15:14:52.174189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 505/1481] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:14:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:14:52.248987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:14:53.008830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:14:53.009098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:14:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:14:53.151384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:53.152092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:53.160573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.161128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:53.162321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.162899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:53.228301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:14:53.236007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b92c53a2-0120-44fb-9076-45598942cc89 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091078 Jul 20 15:14:53.236923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 506/1482] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:14:53 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:14:53.241842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:14:53.242759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:53.248906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.249343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:53.250095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.250517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:53.362148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c5827347-d675-49a6-bf83-90441853268e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action" status: 200 len: 76133 microversion: 2.1 time: 0.123765 Jul 20 15:14:53.362884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 507/1483] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:14:53 2022] POST /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96/action => generated 76133 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:14:53.369397 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:53.369978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:53.377183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.377632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:53.378400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:53.378831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:56.998855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:14:57.001288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:14:57.003454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:14:57.004566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:14:57.006044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:57.007856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:14:57.009257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) Jul 20 15:14:57.010687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:14:57.012214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-8f843865-ee7c-4576-9b93-3d58fcb40b5b) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:14:57.012214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-abb4b555-bf50-4f81-8d8a-4e3aa8ed2a97 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 63.735785 Jul 20 15:14:57.012214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 977/1484] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:13:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 63738 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:14:57.013373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:57.014014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:57.021163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:57.021577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:57.022356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:57.022781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:57.093301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:14:57.100906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32a6a176-e3b0-4e19-b0c1-92a47601dcc0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095490 Jul 20 15:14:57.101463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 978/1485] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:14:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:14:57.215151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:14:58.122270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:58.123846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:58.137967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:58.138601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:58.139779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:58.140418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:58.220644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:14:58.229133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad1c3985-bd7f-4ab7-9938-5ab34cf16d3a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.113844 Jul 20 15:14:58.229682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 979/1486] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:14:58 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:14:59.250558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:14:59.252071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:14:59.263782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:59.264522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:59.265920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:14:59.266511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:14:59.330991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:14:59.338956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8795b791-5d68-45f3-b58b-8f28b31125e8 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095185 Jul 20 15:14:59.339561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 980/1487] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:14:59 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:15:00.359552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:00.360433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:00.370733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:00.371385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:00.372433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:00.373046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:00.436865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:00.444796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bd74d59b-bc9e-46ce-b610-d5e7c80c78e0 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090754 Jul 20 15:15:00.445365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 981/1488] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:00 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:01.466681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:01.467798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:01.479740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:01.480497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:01.481763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:01.482448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:01.548583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:01.556478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff704a51-2ea6-44aa-a8c7-3523d489e9fd tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096519 Jul 20 15:15:01.557125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 982/1489] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:01 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:02.575333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:02.575984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:02.583172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:02.583658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:02.584457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:02.584909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:02.656869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:02.667721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-18fe8dd5-9836-4a77-9a95-9b9483cd1207 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096996 Jul 20 15:15:02.668313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 983/1490] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:02 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:03.374611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=10 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:03.688339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:03.688956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:03.699550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:03.700231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:03.701435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:03.702097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:03.770759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:03.778975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b496c662-adf6-412c-a382-34ed4b737fbc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096392 Jul 20 15:15:03.779629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 984/1491] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:03 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:04.798438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:04.798998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:04.806786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:04.807277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:04.808040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:04.808474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:04.893957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:04.904102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3720a0db-b153-4131-9b05-dc081199b691 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.111058 Jul 20 15:15:04.904962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 985/1492] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:04 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:05.929083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:05.930157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:05.937205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:05.937654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:05.938407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:05.938834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:06.000289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:06.008326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-35f60156-b35e-4620-a9c0-6cfe6d8b0c5a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.085253 Jul 20 15:15:06.009073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 986/1493] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:05 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:07.029594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:07.030605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:07.041102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:07.041576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:07.042336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:07.042772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:07.117935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:07.129772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02dc4622-435e-40df-a7e1-94b6dbb53392 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.106694 Jul 20 15:15:07.130588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 987/1494] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:07 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:07.220218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=30 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:08.152203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:08.152977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:08.161985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:08.162576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:08.163723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:08.164347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:08.231202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:08.238921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-444c1036-721c-4d73-abcb-57a01c2e7ac5 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.092972 Jul 20 15:15:08.239548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 988/1495] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:08 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:09.260304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:09.261067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:09.268538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:09.269034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:09.269873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:09.270345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:09.332862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:09.343879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84cd68b5-76cd-4e6c-9cb7-82ad60a7af65 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089499 Jul 20 15:15:09.344760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 989/1496] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:09 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:10.368810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:10.369490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:10.376625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:10.377081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:10.377851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:10.378286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:10.448263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:10.456346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3f8092ec-394d-44eb-b212-810d4b1f0b77 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094664 Jul 20 15:15:10.456902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 990/1497] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:10 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:11.478838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:11.479565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:11.487308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:11.487736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:11.488509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:11.488947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:11.552062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:11.559945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da264145-9968-4178-aa3a-53bef1136922 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088310 Jul 20 15:15:11.560507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 991/1498] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:11 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:12.581173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:12.582226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:12.590787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:12.591268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:12.592032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:12.592463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:12.654677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:12.662522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b6f106ae-aa8b-4c2c-b9e1-704bf3841ae3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087682 Jul 20 15:15:12.663017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 992/1499] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:12 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:13.683116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:13.684148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:13.695222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:13.695966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:13.697162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:13.697696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:13.700247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=95 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:13.769783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:13.777833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7b296a16-bace-4c5d-983d-bb3cf9115cda tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.101225 Jul 20 15:15:13.778397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 993/1500] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:13 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:14.799459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:14.800570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:14.809723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:14.810178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:14.810963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:14.811423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:14.881838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:14.889746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bf42cee1-c5c7-4df6-8055-bcb8c37510ab tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.096771 Jul 20 15:15:14.890310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 994/1501] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:14 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:15.911020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:15.912034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:15.923197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:15.923942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:15.925201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:15.925922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:15.997344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:16.009778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1473f3e6-3632-430f-82b0-a5a49eb92254 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.105218 Jul 20 15:15:16.010340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 995/1502] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:15 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:17.032141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:17.033255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:17.043015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:17.043498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:17.044231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:17.044680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:17.106910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:17.114661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-761c3990-ce41-4b25-b81d-9507fd01f77b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.088938 Jul 20 15:15:17.115251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 996/1503] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:17 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:18.135595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:18.136651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:18.147782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:18.148524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:18.149936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:18.150365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:18.218081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:18.226123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-499dbe35-e7f8-4c17-ab9f-0292ae1bc70f tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097219 Jul 20 15:15:18.226675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 997/1504] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:18 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:19.248808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:19.249996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:19.261538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:19.261974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:19.262755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:19.263218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:19.328151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:19.338902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c11f527f-148a-4983-9746-bf74c721296c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098049 Jul 20 15:15:19.339491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 998/1505] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:19 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:15:20.359796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:20.360808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:20.370723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:20.371137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:20.371898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:20.372326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:20.444770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:20.452857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a1919676-ee33-4e8e-bbfa-ec8fe799c631 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.099476 Jul 20 15:15:20.453414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 999/1506] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:20 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:21.474022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:21.475202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:21.483144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:21.483581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:21.484344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:21.484772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:21.548673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:21.556731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-12bbc63e-0880-4244-8eea-932d44da77ad tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089292 Jul 20 15:15:21.557259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1000/1507] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:21 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:15:22.582981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:22.584219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:22.592336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:22.592822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:22.593589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:22.594020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:22.659444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:22.667353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-38b5a3bb-6e2e-4371-af33-b8b69b5af852 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.091116 Jul 20 15:15:22.667912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1001/1508] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:22 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:23.693614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:23.694606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:23.697753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=49 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:23.703637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:23.704106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:23.704897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:23.705353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:23.768127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:23.776036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-69aafe7d-7832-4aee-854c-4846418df635 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089074 Jul 20 15:15:23.776596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1002/1509] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:23 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:24.797320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:24.798392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:24.810480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:24.811176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:24.812306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:24.812943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:24.901098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:24.913849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b11de345-83b9-4910-aa3c-f414602f30d7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.122814 Jul 20 15:15:24.914640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1003/1510] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:24 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:25.940920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:25.941992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:25.950614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:25.951082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:25.951856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:25.952292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:26.015506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:26.023916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-991bf98d-7bbc-473c-ba36-b8ba5ebe4423 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.089560 Jul 20 15:15:26.024509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1004/1511] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:25 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:27.045447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:27.046224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:27.056975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:27.057429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:27.058240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:27.058687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:27.132364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:27.144655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36cce0da-fa80-4054-bd91-215f8a8d582c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.105755 Jul 20 15:15:27.145418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1005/1512] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:27 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:28.166263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:28.167116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:28.175692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:28.176251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:28.177192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:28.177732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:28.242193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:28.249941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebd5810e-cfe9-468d-a84a-a73e7b14b55c tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090486 Jul 20 15:15:28.250508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1006/1513] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:28 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:29.271422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:29.272438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:29.280406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:29.280856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:29.281613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:29.282044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:29.354544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:29.366265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7fce5c56-2904-4ac7-aa97-1c7f6d35d6ec tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.101097 Jul 20 15:15:29.367115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1007/1514] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:29 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:30.388333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:30.389257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:30.398266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:30.398918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:30.400082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:30.400720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:30.476938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:30.485399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-712a7dbb-d405-4c6b-acc0-b025608e94d4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.103256 Jul 20 15:15:30.487249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1008/1515] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:30 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:31.503844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:31.504497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:31.511457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:31.511907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:31.512676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:31.513112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:31.580594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:31.582789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:31.589646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c42789a1-ad3b-4714-9565-893d075248e9 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090031 Jul 20 15:15:31.590210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1009/1516] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:31 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:32.607972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:32.608632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:32.615390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:32.615843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:32.616602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:32.617035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:32.681017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:32.688674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86c7e5fa-1949-4aeb-94bc-ded274f34a74 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.084662 Jul 20 15:15:32.689260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1010/1517] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:32 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:33.710318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:33.711415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:33.719233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:33.719725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:33.720558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:33.721037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:33.795649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:33.803987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-29929ed8-e442-47ec-a1bf-47ea393bfb2a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.100054 Jul 20 15:15:33.804560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1011/1518] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:33 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:34.823582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:34.824346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:34.831889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:34.832376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:34.833322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:34.833793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:34.897078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:34.904714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-136a749c-b195-4031-832b-506b281473db tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086859 Jul 20 15:15:34.905281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1012/1519] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:34 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:35.925810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:35.926931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:35.935202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:35.935650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:35.936404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:35.936835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:36.005735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:36.013329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b258c557-8b82-4a27-b249-8e8f395e2f1a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094165 Jul 20 15:15:36.013903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1013/1520] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:35 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:37.030453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:37.031307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:37.039403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:37.041263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:37.042063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:37.042503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:37.105061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:37.112882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4770bdc2-46b0-4d17-8309-cee848a5526a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086628 Jul 20 15:15:37.113457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1014/1521] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:37 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:38.134555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:38.135691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:38.146657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:38.147458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:38.148689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:38.149560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:38.225935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:38.238625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7a01311-efbd-43c4-9681-9fcb54d19f6b tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.110391 Jul 20 15:15:38.239513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1015/1522] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:38 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:39.261087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:39.261911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:39.270329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:39.270894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:39.271886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:39.272430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:39.597906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:39.610655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-baa8ce1b-ae04-46fd-8d74-e7e28c6e4ce4 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.355129 Jul 20 15:15:39.611392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1016/1523] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:39 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:40.634066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:40.635212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:40.646077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:40.646825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:40.648130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:40.648856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:40.713282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:40.725153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-354c5f8b-ef2b-4904-a195-421b9996da1d tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.097502 Jul 20 15:15:40.725716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1017/1524] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:40 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:41.745220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:41.745912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:41.756126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:41.756722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:41.757697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:41.758219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:41.822483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:41.833696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d62b8757-accf-4b61-a77d-0879f00934b7 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.094649 Jul 20 15:15:41.834501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1018/1525] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:41 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:42.852218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:42.852950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:42.859967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:42.860415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:42.861168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:42.861597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:42.922165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:42.930074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d54775b-20dc-40ad-98b7-c0b470121640 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.081862 Jul 20 15:15:42.930689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1019/1526] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:42 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:43.951123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:43.952202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:43.963394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:43.964135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:43.965380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:43.966096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:44.035721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:44.043474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2a543ee0-ad35-4f5e-bd8a-ab07f5079be3 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098909 Jul 20 15:15:44.044038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1020/1527] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:43 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:45.061262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:45.062585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:45.070515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:45.070989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:45.071784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:45.072208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:45.134488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:45.142531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-69652041-1a34-455b-8de4-d30c86a10171 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.085205 Jul 20 15:15:45.143086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1021/1528] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:45 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:46.160757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:46.161585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:46.168663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:46.169091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:46.169862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:46.170293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:46.236824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:46.244636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3338aa1c-ee6c-47de-9163-a7a744cdb3ff tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.087815 Jul 20 15:15:46.245212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1022/1529] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:46 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:47.265861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:47.266913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:47.276648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:47.277094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:47.277859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:47.278303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:47.350080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:47.358256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4a6d209-b319-4a6f-8203-64ef1fd07d53 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.098829 Jul 20 15:15:47.358828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1023/1530] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:47 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:48.380537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:48.382237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:48.389732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:48.390187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:48.390949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:48.391413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:48.460282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:48.468699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6afa2925-9984-4e2d-858a-478fb36b6f4e tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095958 Jul 20 15:15:48.469286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1024/1531] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:48 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:49.490154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:49.491197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:49.498897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:49.499415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:49.500253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:49.501034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:49.562303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:49.570399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4fccc981-8b58-4830-a81a-0afb72a60aea tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.086949 Jul 20 15:15:49.570963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1025/1532] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:49 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:50.596077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:50.597120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:50.605949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:50.606401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:50.607137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:50.607589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:50.622349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:15:50.673499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:50.685118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-234ae6a5-45c6-45d6-bd2f-ef050be4be03 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095197 Jul 20 15:15:50.685879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1026/1533] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:50 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:51.708641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:51.709271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:51.716639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:51.717073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:51.717811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:51.718232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:51.918033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:51.926003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7eb0e195-8baf-4a0c-8ed2-8b20a92fcf61 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.223596 Jul 20 15:15:51.926679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1027/1534] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:51 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:52.947619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:52.948417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:52.956950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:52.957503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:52.958451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:52.958993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:53.028745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:15:53.036717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7be7b8a5-a9c8-44c1-8532-8866e121b3bc tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.095272 Jul 20 15:15:53.037285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1028/1535] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:15:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:15:53.390783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:15:53.391399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:15:53.397828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:53.398278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:15:53.399170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:15:53.399610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:03.395377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=55 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:03.417761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=19 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:16:24.489997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:16:24.490786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:16:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:16:28.234486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:16:28.235631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:16:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:16:40.911433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:16:40.912386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:16:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:16:52.679503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:16:52.682152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:16:52.683641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:16:52.684406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:16:52.685565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:52.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:16:52.690698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) Jul 20 15:16:52.692311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:16:52.693850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-2dda2bbb-9933-45b1-9180-92d42e4f29f6) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:16:52.693850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-00fe2d70-fc6a-4e65-aeae-14971a193589 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 119.319164 Jul 20 15:16:52.693850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 508/1536] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:14:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 119321 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:16:52.758072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:52.758696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:52.765025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:52.765514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:52.766346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:52.766797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:52.830213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:16:52.837915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2783b73e-6d73-425d-947e-0f51c31d7361 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.152626 Jul 20 15:16:52.838513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 509/1537] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:16:52 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:16:53.450382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:53.450889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:53.457588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:53.458048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:53.458815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:53.459380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:53.464890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:16:57.317520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:16:57.319847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:16:57.321951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:16:57.323129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:16:57.324526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:57.326236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:16:57.327711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) Jul 20 15:16:57.329064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:16:57.330375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-3edc5f14-1afe-4325-ae8b-3c6936d5273b) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:16:57.330375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-422476c0-8f3e-457e-9ff4-a7ca0808b425 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 63.934745 Jul 20 15:16:57.330375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1029/1538] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:15:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 63936 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:16:57.330375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:57.330745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] [instance: f30068fe-59db-4a1f-9ad9-040b4e98f342] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:57.335680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.336048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.336936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.337295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.368341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:16:57.405186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '58016f5cae3d45eab6f9caf161e65c4c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '779e3cc39cf14719ba34b3d12cac5a01', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:16:57.413518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-43c6497d-a701-43d7-90cf-466c02808df2 tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342" status: 200 len: 1220 microversion: 2.1 time: 0.090667 Jul 20 15:16:57.416075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1030/1539] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:16:57 2022] GET /compute/v2.1/servers/f30068fe-59db-4a1f-9ad9-040b4e98f342 => generated 1220 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:16:57.472867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:57.473624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:57.480595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.481316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.482755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.483745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.511799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:16:57.630800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d14d9547-4c37-4d4d-b7a9-b297a29114f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 204 len: 0 microversion: 2.1 time: 0.162656 Jul 20 15:16:57.634728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1031/1540] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:16:57 2022] DELETE /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:16:57.641000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb3c157c-8f1a-4a79-9d26-91090d64dd7a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] Calling method '.version_select at 0x7f7b15de5ca0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:57.677006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb3c157c-8f1a-4a79-9d26-91090d64dd7a tempest-TestShelveInstance-547928220 tempest-TestShelveInstance-547928220-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1661908460" status: 202 len: 0 microversion: 2.1 time: 0.040784 Jul 20 15:16:57.677907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1032/1541] 10.222.0.15 () {58 vars in 1253 bytes} [Wed Jul 20 15:16:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1661908460 => generated 0 bytes in 42 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:16:57.683556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a8eb3c41-ae73-4fdf-9a0f-66bf15998aba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] Calling method '.version_select at 0x7f7b15bd7e50>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:57.715126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a8eb3c41-ae73-4fdf-9a0f-66bf15998aba tempest-TestVolumeBootPattern-822710001 tempest-TestVolumeBootPattern-822710001-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1854191454" status: 202 len: 0 microversion: 2.1 time: 0.035557 Jul 20 15:16:57.715731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1033/1542] 10.222.0.15 () {58 vars in 1262 bytes} [Wed Jul 20 15:16:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1854191454 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:16:57.720401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:57.721158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:57.727361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.727804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.728556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:57.728983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:57.875444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:16:57.883994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c3fbac5-6a64-4d9c-b9ab-9cff0aaf0a46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167206 Jul 20 15:16:57.884775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1034/1543] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:16:57 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:16:58.901998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:16:58.902727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:16:58.909702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:58.910150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:58.910927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:16:58.911389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:16:59.048813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:16:59.057430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c7c3c23-8f06-4d35-b5c8-6d674f8b432f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.159399 Jul 20 15:16:59.058034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1035/1544] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:16:58 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:00.080069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:00.080906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:00.087518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:00.087873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:00.088622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:00.089051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:00.225443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:00.233769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61182277-c505-46ee-a06b-23d334c6b682 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.157511 Jul 20 15:17:00.234345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1036/1545] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:01.250615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:01.252174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:01.259722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:01.260679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:01.261924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:01.262717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:01.455652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:01.467279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d628daf7-0d7a-467e-b12c-5f530610b56b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.221130 Jul 20 15:17:01.471308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1037/1546] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:01 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:02.493378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:02.494007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:02.503436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:02.503880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:02.504737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:02.505168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:02.813161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:02.821318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cc6007c5-1d83-48dc-aa34-bf64c49b4a10 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.337479 Jul 20 15:17:02.823801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1038/1547] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:02 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:03.454687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=4 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:03.839764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:03.840382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:03.851909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:03.856402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:03.857447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:03.858046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:04.108790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:04.117313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb687cea-6b2a-46c4-9316-85a0112e050d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.281923 Jul 20 15:17:04.118115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1039/1548] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:03 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:05.134941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:05.135697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:05.144638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:05.145322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:05.146352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:05.147109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:05.488344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:05.498868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a3d35a3a-61ef-4862-86ec-35fc0cfc00db tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.367621 Jul 20 15:17:05.501941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1040/1549] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:05 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:06.518089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:06.518711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:06.526540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:06.527399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:06.528186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:06.528596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:06.767301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:06.777512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b10ae85-4bd0-4cf8-8ee0-1a5878e2e3b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.263399 Jul 20 15:17:06.778842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1041/1550] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:06 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:07.381267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=14 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:07.795031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:07.795652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:07.802245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:07.802789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:07.803630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:07.804390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:07.969359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:07.979459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a536b227-a515-423a-94a2-e0255285fb1e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.188168 Jul 20 15:17:07.980024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1042/1551] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:07 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:09.006579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:09.008192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:09.023451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:09.024188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:09.025451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:09.026190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:09.171080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:09.179521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-26cf84e3-8a8c-406c-a3a4-d9a73b519b9e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.183496 Jul 20 15:17:09.180100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1043/1552] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:08 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:10.204594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:10.205616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:10.216514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:10.217260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:10.218501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:10.219285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:10.359410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:10.367949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b54d4c3-cf0c-4ca4-8431-c47001c9a36e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169802 Jul 20 15:17:10.368525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1044/1553] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:10 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:11.390319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:11.391366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:11.400567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:11.400943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:11.401760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:11.402187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:11.537700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:11.545914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-231d53e7-64d4-4e0a-9bfe-4f63e32c178f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162014 Jul 20 15:17:11.546481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1045/1554] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:11 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:12.565517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:12.566388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:12.576108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:12.576782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:12.577820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:12.578440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:12.584533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:12.713809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:12.722363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fda5af62-cf05-42e1-aea5-caf331976984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162931 Jul 20 15:17:12.722938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1046/1555] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:12 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:13.749433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:13.750435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:13.759649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:13.760094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:13.760841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:13.761265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:13.903625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:13.912282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-401988d5-d7ac-4e90-993d-1e25c7e49042 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170548 Jul 20 15:17:13.912834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1047/1556] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:13 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:14.933769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:14.934634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:14.943156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:14.943705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:14.944643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:14.945174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:15.085707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:15.095893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-46daabe0-1f41-4df0-8dfd-2d08d31fd65d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168437 Jul 20 15:17:15.096806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1048/1557] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:14 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:16.120260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:16.121360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:16.132195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:16.132634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:16.133385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:16.133805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:16.267782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:16.276158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-749bf3a3-7ac4-485c-b6d7-91ec9b8460a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162611 Jul 20 15:17:16.276738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1049/1558] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:16 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:17.297587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:17.298682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:17.307660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:17.308112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:17.308877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:17.309309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:17.451784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:17.462749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c08e6703-b528-4a32-9978-3b607c6f9ccc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.171504 Jul 20 15:17:17.463581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1050/1559] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:17 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:18.480951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:18.481633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:18.488476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:18.488936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:18.489704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:18.490137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:18.632560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:18.640932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02b5d713-5f1a-4ea7-8e3d-51e68d293ef5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163876 Jul 20 15:17:18.641499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1051/1560] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:18 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:19.660600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:19.661286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:19.671396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:19.672038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:19.673173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:19.673794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:19.823739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:19.832322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-db5c0930-ff14-41d5-9520-5da54c3f2bcd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.177113 Jul 20 15:17:19.832887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1052/1561] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:19 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:20.858453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:20.859487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:20.868825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:20.869290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:20.870080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:20.870528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:21.006985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:21.015439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-204c6d05-e0c6-42da-9778-c04f86a42ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163767 Jul 20 15:17:21.016008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1053/1562] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:20 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:22.036844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:22.037844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:22.044712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:22.045153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:22.045897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:22.046323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:22.188357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:22.196565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81f1005b-a538-4e4d-94a4-51d0f6a0d942 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166481 Jul 20 15:17:22.197139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1054/1563] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:22 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:23.216860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:23.217874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:23.229182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:23.229951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:23.231303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:23.232024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:23.373089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:23.381554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bcfec5e2-9ece-4e30-9c64-81efa545397f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.171041 Jul 20 15:17:23.382158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1055/1564] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:23 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:17:24.410652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:24.412588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:24.415678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=60,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:24.421369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:24.421817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:24.422579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:24.423012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:24.559892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:24.572501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b8720d20-dbf3-4dfa-b748-650c88cea68a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172187 Jul 20 15:17:24.573314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1056/1565] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:24 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:25.597856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:25.598821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:25.609694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:25.610423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:25.611885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:25.612599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:25.744374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:25.753110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d4baab8-5b00-4cc6-8f3a-98a60ab175d0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161988 Jul 20 15:17:25.753678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1057/1566] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:25 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:26.773935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:26.774960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:26.787489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:26.787932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:26.788718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:26.789146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:26.927008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:26.935903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40ed3ac7-4725-4e9b-ac35-fea717f7cb63 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168576 Jul 20 15:17:26.936451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1058/1567] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:26 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:27.957917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:27.958495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:27.965407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:27.965852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:27.966626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:27.967025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:28.100302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:28.108943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0b0a3bf8-b25a-4a72-8099-442d2d0edcda tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.154903 Jul 20 15:17:28.109538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1059/1568] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:27 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:29.128356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:29.129292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:29.138447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:29.139038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:29.140083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:29.140662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:29.275278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:29.284072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-012ed1b2-e6b8-438f-8749-840b80aaf9dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161320 Jul 20 15:17:29.284659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1060/1569] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:29 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:30.305113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:30.306189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:30.316610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:30.317340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:30.318597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:30.319028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:30.465922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:30.474185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f6bd8e1-8037-449b-ae5a-f4489786fe82 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.175520 Jul 20 15:17:30.474840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1061/1570] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:30 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:31.493707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:31.494284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:31.501375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:31.501812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:31.502557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:31.502983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:31.640531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:31.643992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:31.649704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3b8b1f3-2c8f-4c6d-ae1b-1acbb4e4e24c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.160665 Jul 20 15:17:31.650272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1062/1571] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:31 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:32.673979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:32.674732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:32.681919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:32.682427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:32.683346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:32.683774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:32.822874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:32.835196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dca8f777-cf36-49dd-800c-dd1de1798bcb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166464 Jul 20 15:17:32.835758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1063/1572] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:32 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:33.855946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:33.856647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:33.865599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:33.866038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:33.866784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:33.867255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:34.011475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:34.020196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0af9ac13-687f-497a-996c-880bb668aee5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170295 Jul 20 15:17:34.020738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1064/1573] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:33 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:35.037524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:35.038124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:35.045135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:35.045585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:35.046346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:35.046781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:35.179983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:35.188143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-acfdf212-7b46-4656-9638-80da73228e45 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.154619 Jul 20 15:17:35.188712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1065/1574] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:35 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:36.213309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:36.214384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:36.225055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:36.225501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:36.226286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:36.226714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:36.364572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:36.377145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4a8f7991-a790-4e30-b0a1-e772b1a2098f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170127 Jul 20 15:17:36.377923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1066/1575] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:36 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:37.398201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:37.398844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:37.405935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:37.406386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:37.407175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:37.407619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:37.545553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:37.553996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e9c26061-8d7d-4361-a05c-4a6dcb5064cf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161160 Jul 20 15:17:37.554583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1067/1576] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:37 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:38.578760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:38.579664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:38.588974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:38.589558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:38.590601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:38.591242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:38.737138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:38.745961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8ccaa542-e78c-4418-96d3-3ba3d248a42a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172719 Jul 20 15:17:38.746565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1068/1577] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:38 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:39.767789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:39.768967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:39.777187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:39.777633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:39.778388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:39.778819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:39.917761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:39.930351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7b4ab469-a6ea-4942-9301-8ad19259ec67 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168977 Jul 20 15:17:39.931195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1069/1578] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:39 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:40.948233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:40.948948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:40.955726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:40.956174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:40.956915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:40.957343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:41.098754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:41.110910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40f9504d-e546-4241-97d6-4e5e720272b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166596 Jul 20 15:17:41.111714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1070/1579] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:40 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:42.132912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:42.133962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:42.140862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:42.141306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:42.142270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:42.142698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:42.281285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:42.289511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c2914d0-e4e6-400c-88d5-5ed4dfdf4317 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163172 Jul 20 15:17:42.290205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1071/1580] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:43.310735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:43.311881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:43.323020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:43.323829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:43.325065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:43.325773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:43.472239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:43.480576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b7f340f-ccb3-43aa-8873-4055cb3bc71d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176232 Jul 20 15:17:43.481383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1072/1581] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:43 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:44.502222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:44.503275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:44.512441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:44.512890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:44.513653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:44.514088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:44.660010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:44.668659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-448e7a1e-1040-4efc-a056-2e1ec498f6c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173304 Jul 20 15:17:44.669235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1073/1582] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:44 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:45.689246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:45.690357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:45.701119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:45.701859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:45.703124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:45.703850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:45.845442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:45.856005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3529016-b364-4dce-aa5f-9485a3e4fad1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173182 Jul 20 15:17:45.856578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1074/1583] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:45 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:46.873596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:46.874295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:46.881376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:46.881833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:46.882784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:46.883271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:47.026100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:47.037093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1fc6304f-2475-4585-b364-e2becd34b0c1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167708 Jul 20 15:17:47.037657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1075/1584] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:46 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:48.059437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:48.060272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:48.068548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:48.069081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:48.069977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:48.070484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:48.214377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:48.226521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1afd590e-5508-4497-a03c-dcc7d4b9dceb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173133 Jul 20 15:17:48.227514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1076/1585] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:48 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:49.253816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:49.254780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:49.264345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:49.264808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:49.265552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:49.265986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:49.399788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:49.412771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5fab49ca-8217-4365-ba22-57e2d9ae877b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165789 Jul 20 15:17:49.413576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1077/1586] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:50.436649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:50.437701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:50.448789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:50.449548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:50.450778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:50.451538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:50.611221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:50.619962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8fb8d674-8fd9-43e8-bb4e-f652b45db41e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.190322 Jul 20 15:17:50.620544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1078/1587] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:50 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:51.638439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:51.639211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:51.646496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:51.646941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:51.647743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:51.648172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:51.667394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:17:51.798284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:51.806720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fdeb49e9-9b28-4d14-934b-0edd4f89cdd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172292 Jul 20 15:17:51.807343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1079/1588] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:51 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:52.827972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:52.829158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:52.840341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:52.841172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:52.842495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:52.843307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:52.988685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:17:53.001209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe664d21-5be7-4a18-ae3c-821656a898ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.179681 Jul 20 15:17:53.001956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1080/1589] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:17:52 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:17:53.512845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:17:53.513346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:17:53.522599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:53.523301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:17:53.524379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:17:53.524903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:03.518702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=51 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:18:03.547003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=12 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:18:13.019617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:18:13.021721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:18:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b (ip 10.222.0.15) !!! Jul 20 15:18:53.772888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:18:53.775185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:53.777122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:53.778252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:53.779739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:53.781269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:53.782624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) Jul 20 15:18:53.785537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:53.786933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-847e4278-76d7-4ea5-a473-3eeecf42f622) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:18:53.786933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7545912f-946c-4136-9512-75c56b7777c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 120.330270 Jul 20 15:18:53.786933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 510/1590] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:16:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 120332 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 20 15:18:53.786933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:53.787512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:53.792623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:53.793017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:53.793776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:53.794227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:53.928317 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:53.936644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-da010162-347e-4fe9-9b7b-2fc1be57d326 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.158176 Jul 20 15:18:53.937240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 511/1591] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:18:53.942075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:53.942568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:53.948809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:53.949263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:53.950040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:53.950480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:54.162556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:18:54.331725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.objects.instance [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'services' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124397) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:18:54.360450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:18:54.362875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:54.364401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:54.365290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:54.366369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.367691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:54.368945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) Jul 20 15:18:54.370348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.372091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-63f4990d-ac2d-43d0-ac5b-df592516eca0) {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:18:54.372091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d17a5a6f-ef06-474f-afee-a5196846ae9c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 60.857883 Jul 20 15:18:54.372091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1081/1592] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:17:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 60859 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:18:54.395935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.wsgi [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: request: DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b HTTP/1.1 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept: application/json Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Accept-Encoding: identity Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Connection: close Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Length: 0 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Content-Type: application/json Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Host: 10.222.0.15 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Openstack-System-Scope: None Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: User-Agent: python-urllib3/1.26.9 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Id: None Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Domain-Name: None Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Identity-Status: Confirmed Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Is-Admin-Project: True Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Id: default Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Domain-Name: Default Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Role: reader,member Jul 20 15:18:54.398170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Roles: reader,member Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Id: default Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Domain-Name: Default Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:18:54.399807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: method: > Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: action_args: {'server_id': '75115162-3a7f-4a94-a703-190f8fdbc4c8', 'id': '66cea3b7-c23c-43b5-ba14-cdcdf0c9952b'} Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:54.400597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5049, in _detach_volume Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:18:54.401963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.403444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5071, in detach_volume Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5051, in _detach_volume Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:54.405004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) Jul 20 15:18:54.406398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: ERROR nova.api.openstack.wsgi Jul 20 15:18:54.407753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-376f018a-b51d-4354-8486-ec16ec42d957) {{(pid=124397) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:18:54.407753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-817b3c08-45b1-475a-9315-1ca5ef81e4cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b" status: 400 len: 269 microversion: 2.1 time: 0.461437 Jul 20 15:18:54.407753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 512/1593] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:18:53 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments/66cea3b7-c23c-43b5-ba14-cdcdf0c9952b => generated 269 bytes in 462 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:18:54.411241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:54.412403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:54.418864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:54.419345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:54.420112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:54.420546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:54.452827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40085558-e30e-4d1d-aeab-090d7becd8ee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046732 Jul 20 15:18:54.453394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1082/1594] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:54.959647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:54.960600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:54.972024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:54.972488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:54.973266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:54.973700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:55.117153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:55.126074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c3dc2189-7863-4e7d-b4a9-f43ccf22a096 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174615 Jul 20 15:18:55.126648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 513/1595] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:54 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:18:55.470680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:55.472178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:55.479610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:55.480075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:55.480873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:55.481332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:55.513337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57189c5e-0953-47ae-94fc-5ac280140b3d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049608 Jul 20 15:18:55.513892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1083/1596] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:56.147771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:56.148811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:56.160036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:56.160496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:56.161265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:56.161702 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:56.302725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:56.315340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7643da73-7552-48e7-a19f-9462c31b3cc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173980 Jul 20 15:18:56.316131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 514/1597] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:18:56.525583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:56.526733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:56.534801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:56.535382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:56.536142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:56.536574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:56.566875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-77eb94ab-c484-4fb4-8e96-47c3933c80bb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045394 Jul 20 15:18:56.567456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1084/1598] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:56 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:57.341499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:57.342510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:57.351822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:57.352271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:57.353036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:57.353467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:57.488924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:57.501463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e6ebb31f-44d9-4ab0-aa66-af0b9889b2d2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166597 Jul 20 15:18:57.502052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 515/1599] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:57 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:18:57.584462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:57.585990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:57.597421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:57.598048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:57.598809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:57.599281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:57.629672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-591f3889-0ee6-44c3-a5d7-bb97a0f6bdd6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051968 Jul 20 15:18:57.630240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1085/1600] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:58.519891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:58.520497 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:58.527772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:58.528345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:58.529428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:58.529864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:58.641730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:58.642660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:58.650538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:58.650993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:58.651827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:58.652257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:58.677784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:58.683232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce3518da-65b3-4a7f-9142-12239872660c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045592 Jul 20 15:18:58.684073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1086/1601] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:58.686295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d59e28f1-9449-49ac-9e55-8700ab7ef42d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170546 Jul 20 15:18:58.686888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 516/1602] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:58 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:18:59.700916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:59.702498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:59.706966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:18:59.708329 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:18:59.713378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:59.714120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:59.715428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:59.716198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:59.718895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:59.719733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:59.721002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:18:59.721724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:18:59.752039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-842bd8f0-1ee6-422f-949f-c0bba13d441a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058702 Jul 20 15:18:59.752756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1087/1603] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:18:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:18:59.872728 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:18:59.883002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-812588f4-66ed-4c47-bd36-0afef8654e9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.182550 Jul 20 15:18:59.883576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 517/1604] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:18:59 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:00.764943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:00.765847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:00.772894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:00.773358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:00.774154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:00.774583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:00.806342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebee4712-d9bb-4922-b908-e627e1a7e244 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045683 Jul 20 15:19:00.806890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1088/1605] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:00.904975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:00.906305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:00.917456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:00.918194 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:00.919510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:00.920223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:01.063957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:01.072829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7e8c24f9-b43f-4526-9622-2e8870961cf2 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174429 Jul 20 15:19:01.073433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 518/1606] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:01.818811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:01.819796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:01.826975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:01.827481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:01.828254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:01.828763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:01.859915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b8e75ee9-cb26-48c6-897e-b6b334588bcb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045361 Jul 20 15:19:01.860469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1089/1607] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:02.095002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:02.096070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:02.103796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:02.104245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:02.105004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:02.105431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:02.243197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:02.257333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4ef54f2a-5d03-4dd8-8cd2-3ce1943bfc27 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168893 Jul 20 15:19:02.258147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 519/1608] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:02 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:02.876728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:02.878239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:02.890488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:02.891646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:02.893553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:02.894633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:02.938058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-962ad34b-175c-4abd-a7ed-5595555b5c6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068541 Jul 20 15:19:02.938651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1090/1609] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:02 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:03.275334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:03.275920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:03.283216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:03.283678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:03.284435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:03.284897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:03.429873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:03.438231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-28520f6a-c354-4696-b877-d414b3f803e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167816 Jul 20 15:19:03.438787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 520/1610] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:03 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:03.956776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:03.958299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:03.966066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:03.966562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:03.967438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:03.967918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:03.999427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c90becf3-a487-4161-84f7-75ca0e934ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049370 Jul 20 15:19:03.999910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1091/1611] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:04.460351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:04.461443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:04.473048 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:04.473872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:04.475613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:04.475936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:04.627793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:04.636647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bec577bf-0ea9-41a6-a7f2-35d18cf84e8f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.183023 Jul 20 15:19:04.637221 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 521/1612] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:04 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:05.011482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:05.012409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:05.019090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:05.019572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:05.020362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:05.020798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:05.050698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c4f6fa4-0d23-4262-8449-27743ec0ba87 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043286 Jul 20 15:19:05.051584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1092/1613] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:05.658144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:05.659231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:05.667143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:05.667607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:05.668385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:05.668819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:05.813263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:05.822160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4c2b0ee4-db61-4462-8e62-123620c6b26a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170726 Jul 20 15:19:05.822737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 522/1614] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:05 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:06.066604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:06.067773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:06.079006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:06.079677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:06.080814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:06.081447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:06.111953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d178b34f-e16b-410c-8893-bfd0448fef57 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051885 Jul 20 15:19:06.112503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1093/1615] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:06.839872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:06.840405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:06.847198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:06.847635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:06.848394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:06.848819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:07.124717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:07.125694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:07.133259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:07.133708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:07.134483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:07.134919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:07.167166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6aa2739e-b75e-47d5-9fae-26a3279908a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046665 Jul 20 15:19:07.167771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1094/1616] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:07.299831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:07.313046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8a26619c-4cb6-4b5a-95aa-9ab96383aeef tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.476852 Jul 20 15:19:07.316034 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 523/1617] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:06 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:08.181131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:08.181966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:08.189626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:08.190081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:08.190845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:08.191317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:08.229982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27533459-384a-4058-ab8d-83066eb684a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054017 Jul 20 15:19:08.230548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1095/1618] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:08.334262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:08.334827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:08.342078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:08.342548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:08.343611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:08.344045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:08.497737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:08.506184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-39e62ea3-7cd4-48e4-a7c0-eda23fc2dc5c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.177740 Jul 20 15:19:08.506748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 524/1619] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:08 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:09.242035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:09.242974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:09.250249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:09.250700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:09.251456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:09.251924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:09.281782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-594993b3-05b0-4cb4-ae0e-f18faed5f0ec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043729 Jul 20 15:19:09.282344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1096/1620] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:09.524073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:09.524686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:09.532193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:09.532642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:09.533411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:09.533842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:09.669750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:09.678160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0879db81-65a3-497a-b67b-4a4989fa410d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.158013 Jul 20 15:19:09.678732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 525/1621] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:09 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:10.297800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:10.299358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:10.310437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:10.311236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:10.312490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:10.313221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:10.342375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:10.347465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc156c54-d985-4fcd-8f0c-457fd066f055 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056271 Jul 20 15:19:10.347993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1097/1622] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:10.700110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:10.701192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:10.710736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:10.711272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:10.712066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:10.712527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:10.847834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:10.856393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8e736a4c-ff0e-400b-83e0-f0fdb80fc619 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163318 Jul 20 15:19:10.856961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 526/1623] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:10 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:11.363334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:11.364907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:11.372556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:11.372986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:11.373723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:11.374148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:11.405366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-680f3df2-2273-46da-9367-81be05671dc9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049031 Jul 20 15:19:11.405932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1098/1624] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:11.878063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:11.878855 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:11.888649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:11.889203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:11.890368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:11.890763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:11.895541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:12.036146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:12.048506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2a7e1dc9-f4e6-48a0-9999-46bcde7a94bd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176673 Jul 20 15:19:12.049266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 527/1625] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:11 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:12.423647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:12.425128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:12.436097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:12.437023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:12.438253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:12.438941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:12.469815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-deb35b5f-4f73-4aac-9c49-0a8130533eda tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053194 Jul 20 15:19:12.470365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1099/1626] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:13.073719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:13.074683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:13.085827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:13.086618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:13.087945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:13.088662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:13.230690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:13.243524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c0729db9-876f-4727-94cd-daaca4dad7a0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176175 Jul 20 15:19:13.244336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 528/1627] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:13 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:13.486922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:13.488550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:13.496477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:13.496925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:13.497690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:13.498124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:13.530569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-714cb1a2-dc70-4c8a-8fe9-127c92f3837a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050256 Jul 20 15:19:13.531374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1100/1628] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:14.267430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:14.268386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:14.276407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:14.276850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:14.277603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:14.278028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:14.418767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:14.427774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ea10f252-d2c7-42dc-b750-e87bb1264f23 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167078 Jul 20 15:19:14.428344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 529/1629] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:14 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:14.546467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:14.547982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:14.558927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:14.559727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:14.560970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:14.561686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:14.596594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-861eb88e-1c00-41cd-b492-c014c14f1412 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056902 Jul 20 15:19:14.597156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1101/1630] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:15.448507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:15.449435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:15.460601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:15.461350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:15.462595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:15.463367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:15.607743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:15.608817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:15.609750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:15.616068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-36c579a1-71a1-45f5-be75-46724d4f9ad6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174308 Jul 20 15:19:15.616599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:15.616768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 530/1631] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:15 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:15.617049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:15.617810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:15.618245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:15.648388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f77a7275-b5ad-42b0-90a5-30d10b6d1ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043465 Jul 20 15:19:15.648960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1102/1632] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:15 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:16.636073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:16.636968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:16.646664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:16.647108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:16.647893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:16.648325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:16.660483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:16.661434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:16.668492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:16.668946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:16.669712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:16.670148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:16.700693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-db42b4ad-38ac-41ff-8301-4e703eee7ed0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044231 Jul 20 15:19:16.701518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1103/1633] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:16.782379 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:16.794019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd93fe80-7558-4bfc-858d-6c2230fc0503 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163340 Jul 20 15:19:16.794766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 531/1634] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:16 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:17.713912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:17.714861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:17.722219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:17.722661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:17.723443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:17.723870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:17.755414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b20b90b0-7335-46ac-8b44-c37906437bc7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045327 Jul 20 15:19:17.755892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1104/1635] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:17 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:17.817538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:17.818672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:17.829848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:17.830588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:17.831901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:17.832619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:17.965764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:17.974405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ef5b907b-6653-4b34-97b7-df366f6db44e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163379 Jul 20 15:19:17.974983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 532/1636] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:17 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:18.768821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:18.769810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:18.777923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:18.778466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:18.779429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:18.779941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:18.810677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-acb155d1-e424-4502-9cdc-88614c17ae4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046726 Jul 20 15:19:18.811244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1105/1637] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:19.001156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:19.002446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:19.014841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:19.015620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:19.016897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:19.017658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:19.152058 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:19.160451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4a235f3c-bb62-459b-9fcf-d8f3bfc2ca44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166442 Jul 20 15:19:19.161028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 533/1638] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:18 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:19.826872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:19.828399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:19.839454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:19.840203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:19.841444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:19.842233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:19.874546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-339fef74-4404-49f4-8c5c-7faa5e3881b3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054381 Jul 20 15:19:19.875087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1106/1639] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:20.185092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:20.185972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:20.196652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:20.197371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:20.198607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:20.199382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:20.341853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:20.353022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f9f436a4-d11f-496a-beab-bc1768a4f514 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174504 Jul 20 15:19:20.353910 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 534/1640] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:20 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:20.890201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:20.891723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:20.900120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:20.900567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:20.901309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:20.901736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:20.931341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6d793acf-7d17-4699-a0a1-ed109700b369 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048056 Jul 20 15:19:20.931872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1107/1641] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:21.379109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:21.380176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:21.387371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:21.387825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:21.388588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:21.389022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:21.523954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:21.532240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-87471b6c-8951-4757-9d55-7eed3b751c44 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.160035 Jul 20 15:19:21.532837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 535/1642] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:21 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:21.943088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:21.944033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:21.950803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:21.951290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:21.952158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:21.952583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:21.982204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-273c39fd-21d7-4e23-971f-d16bc6b9652a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043113 Jul 20 15:19:21.982743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1108/1643] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:21 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:22.550288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:22.550909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:22.557693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:22.558150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:22.558926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:22.559409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:22.694285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:22.706672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8fc1731d-7821-4c17-b098-18cc3149987c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.160175 Jul 20 15:19:22.707388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 536/1644] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:22 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:22.999657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:23.000894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:23.009462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:23.009906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:23.010697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:23.011158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:23.040411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e2dc44b5-a407-4365-b376-199fc9cf75a5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047697 Jul 20 15:19:23.040960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1109/1645] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:22 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:23.728380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:23.729343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:23.733016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=53 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:23.742209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:23.742834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:23.744186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:23.744897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:23.893248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:23.903219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-73c2cd43-a710-4417-b26b-f4506acdf930 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.181442 Jul 20 15:19:23.903820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 537/1646] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:23 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:24.054763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:24.055727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:24.058045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=57 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:24.063967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:24.064416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:24.065159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:24.065592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:24.095761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6f55ece7-1d83-4c3a-b14c-77f8f1fb6621 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046425 Jul 20 15:19:24.096315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1110/1647] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:24.924434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:24.925402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:24.933347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:24.933787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:24.934539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:24.934965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:25.066541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:25.074819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d6292b8b-2b45-4cb9-91a6-3ce5a0d56414 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.156844 Jul 20 15:19:25.075478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 538/1648] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:24 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:25.107734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:25.108682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:25.115577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:25.116026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:25.116801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:25.117215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:25.147329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-46438f39-f5fc-486c-80b4-af1463e2d157 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043632 Jul 20 15:19:25.147879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1111/1649] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:26.096269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:26.097255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:26.105343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:26.105791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:26.106541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:26.106967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:26.161533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:26.162544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:26.170345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:26.170799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:26.171613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:26.172056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:26.202405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-15feb59f-4218-4dce-abdf-f8c7d3f9550d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045034 Jul 20 15:19:26.202980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1112/1650] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:26.249030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:26.259338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f77968f2-ffe5-4480-a41c-369e2ed95c92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169648 Jul 20 15:19:26.260186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 539/1651] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:26 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:27.219021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:27.220595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:27.230025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:27.230488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:27.231316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:27.231759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:27.262951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72b71645-1c73-4db8-bb73-f238ac8c4ba7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051108 Jul 20 15:19:27.263556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1113/1652] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:27 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:27.282022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:27.282643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:27.289597 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:27.290077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:27.290832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:27.291294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:27.426902 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:27.435501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dde25095-3292-4729-a76f-4e5eb45b0a7b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.157461 Jul 20 15:19:27.436072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 540/1653] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:27 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:28.279092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:28.280365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:28.290252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:28.290701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:28.291492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:28.291920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:28.308682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:28.322309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85b6187e-bdad-41ca-88ce-6f6c86028318 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050168 Jul 20 15:19:28.322852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1114/1654] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:28.457069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:28.458095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:28.466080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:28.466545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:28.467355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:28.467793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:28.615706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:28.625019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-245795d6-6091-4fe3-9bf3-6600b0f50955 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174748 Jul 20 15:19:28.625635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 541/1655] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:28 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:29.338192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:29.339139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:29.355854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:29.356316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:29.357098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:29.357529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:29.389739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ba81f74-0fbd-4e04-9bd3-50b7be6ac5b6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057946 Jul 20 15:19:29.390293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1115/1656] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:29.646238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:29.647217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:29.658784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:29.659585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:29.660845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:29.661568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:29.815972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:29.824416 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c2d35667-ce5c-4bf6-8da8-81a50e205ddd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.184426 Jul 20 15:19:29.824986 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 542/1657] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:29 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:30.406010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:30.407583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:30.417693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:30.418148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:30.418897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:30.419359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:30.448655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-de9178cc-2cce-4517-b68c-5ccecd88d978 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049403 Jul 20 15:19:30.449266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1116/1658] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:30.844387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:30.845149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:30.853770 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:30.854339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:30.855343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:30.855962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:30.999925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:31.004879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:31.012856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-919662a7-c4e0-486b-bc81-bbb220d9483f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173134 Jul 20 15:19:31.013438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 543/1659] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:30 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:31.464786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:31.466150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:31.473680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:31.474133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:31.474905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:31.475375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:31.507446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc462fc6-eeba-46ab-8e9f-a3cd9faefd4d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049317 Jul 20 15:19:31.508016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1117/1660] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:31 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:32.032158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:32.032745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:32.039954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:32.040430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:32.041247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:32.041705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:32.188088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:32.196598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2a27b26f-b53d-4f15-a8f0-36b5ed510176 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169592 Jul 20 15:19:32.197172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 544/1661] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:32 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:32.519319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:32.520876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:32.531426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:32.531864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:32.532619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:32.533045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:32.564889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-698e6897-ce7c-4fdc-9542-1ed3503cd76f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049556 Jul 20 15:19:32.565402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1118/1662] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:32 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:33.217836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:33.218929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:33.229945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:33.231198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:33.231762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:33.232205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:33.372977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:33.381294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-eebf122f-f5a1-4181-970d-de306b7a331f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169922 Jul 20 15:19:33.381860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 545/1663] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:33 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:33.582567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:33.584104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:33.594420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:33.594864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:33.595947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:33.596517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:33.634560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c24cdf29-6c73-48d3-b263-d3752fc02b63 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058449 Jul 20 15:19:33.635151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1119/1664] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:34.403294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:34.404061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:34.412638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:34.413203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:34.414157 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:34.414698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:34.562620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:34.571013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d87f3f79-e9e6-4e75-bae3-6aee3603647b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174608 Jul 20 15:19:34.571995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 546/1665] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:34 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:34.649678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:34.651168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:34.662282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:34.662921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:34.664226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:34.664946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:34.696476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5b032cf9-6ce9-4944-ab53-c988be9036f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053541 Jul 20 15:19:34.697042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1120/1666] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:34 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:35.589264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:35.589882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:35.597072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:35.597517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:35.598281 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:35.598713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:35.708725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:35.709669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:35.716804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:35.717291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:35.718064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:35.718492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:35.741268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:35.749815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c021990e-8c82-4d90-8c43-5e2214a7848e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045216 Jul 20 15:19:35.750055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd63defa-bb76-4f76-bf22-4e9de65ff842 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164610 Jul 20 15:19:35.750378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1121/1667] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:35 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:35.750554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 547/1668] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:35 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:36.766175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:36.767592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:36.771268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:36.772292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:36.778950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:36.779461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:36.780252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:36.780690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:36.781891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:36.782347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:36.783145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:36.783573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:36.813012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-657e14c3-80ee-4ac1-b6af-857fb59f7ec4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053821 Jul 20 15:19:36.813846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1122/1669] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:36.921835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:36.930625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9fc484f8-2949-4bf8-919f-ba678decb76c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165969 Jul 20 15:19:36.931232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 548/1670] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:36 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:37.828315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:37.829286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:37.836807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:37.837288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:37.838060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:37.838495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:37.869385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-619e84a5-af95-43b7-bf90-a70a63fd6bc0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046656 Jul 20 15:19:37.870296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1123/1671] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:37 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:37.948399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:37.949000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:37.955922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:37.956372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:37.957134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:37.957568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:38.111006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:38.123709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-50f07022-16d9-43d2-a876-0cd30ed76536 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.179112 Jul 20 15:19:38.124307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 549/1672] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:37 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:38.886943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:38.888479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:38.899920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:38.900654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:38.901905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:38.902609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:38.932438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9202f2e9-f30d-4cc0-a03f-3d363bb20679 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052185 Jul 20 15:19:38.933009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1124/1673] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:39.148371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:39.149284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:39.160701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:39.161481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:39.162744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:39.163516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:39.308942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:39.320453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e84e7f8d-624d-49db-98e1-0a668395091e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.179524 Jul 20 15:19:39.321235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 550/1674] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:39 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:39.948914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:39.950453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:39.958257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:39.958701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:39.959475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:39.960182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:39.991400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-49e82e0d-805c-4df4-9a88-74c84e63fcfd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049242 Jul 20 15:19:39.991981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1125/1675] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:40.343734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:40.344407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:40.352756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:40.353290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:40.354138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:40.354619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:40.498907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:40.507806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3c04988a-68fb-46bf-ac6e-16f648404cdd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170786 Jul 20 15:19:40.508392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 551/1676] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:40 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:41.009630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:41.011463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:41.022755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:41.023558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:41.024786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:41.025502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:41.056799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57b006b1-8a2b-4583-aee0-df07c1329885 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053773 Jul 20 15:19:41.057333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1126/1677] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:41.528283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:41.528939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:41.536484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:41.536983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:41.537832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:41.538322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:41.671247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:41.679895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d4834d73-93c5-4414-85f3-00fab656c728 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.155954 Jul 20 15:19:41.680492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 552/1678] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:41 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:42.072387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:42.073907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:42.082547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:42.083025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:42.083815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:42.084241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:42.114423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-df2e23bd-527a-4289-81c6-0cab2e35eb10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048782 Jul 20 15:19:42.114976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1127/1679] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:42 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:42.702062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:42.703286 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:42.710941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:42.711497 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:42.712347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:42.712806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:42.858886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:42.867549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2abf39f0-3f9e-42b0-9dfd-3388fad287a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172203 Jul 20 15:19:42.868153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 553/1680] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:43.130433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:43.131492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:43.140199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:43.140763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:43.141696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:43.142231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:43.175606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-592a9b14-80f1-4f40-82db-29f6c76c751a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051310 Jul 20 15:19:43.176156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1128/1681] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:43 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:43.741232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:19:43.741509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:19:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:19:43.886552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:43.887276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:43.895531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:43.896095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:43.897001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:43.897511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:44.041070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:44.051444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e777139c-495f-4ced-b595-b0fd0a5ae87d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169093 Jul 20 15:19:44.052289 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 554/1682] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:43 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:44.188099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:44.189082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:44.196456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:44.196938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:44.197729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:44.198177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:44.229183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ead159fd-7c82-42b4-9a37-f22088bd8ea1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045454 Jul 20 15:19:44.229790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1129/1683] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:44.502338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:19:44.502523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:19:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:19:45.078345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:45.079344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:45.087810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:45.088256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:45.089018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:45.089451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:45.242499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:45.243953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:45.247101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:45.253731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:45.254483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:45.255592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:45.256024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:45.258335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ec360c17-c352-43db-a8ae-2b28b9516ef1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.186161 Jul 20 15:19:45.258873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 555/1684] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:45 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:45.286221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2ea4b06-0cc2-4bf5-b53b-70ad0296505e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048563 Jul 20 15:19:45.286820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1130/1685] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:46.278820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:46.279914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:46.289472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:46.289958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:46.290732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:46.291193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:46.298046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:46.298988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:46.305758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:46.306230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:46.306997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:46.307471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:46.311465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:46.338659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ab2fc7d-bbb6-42bd-abc6-69ec0d081a7e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044569 Jul 20 15:19:46.339492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1131/1686] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:46 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:46.425725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:46.434476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2fb67ba5-09d5-4d95-93d7-b17bfca4f163 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162143 Jul 20 15:19:46.435026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 556/1687] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:46 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:47.350535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:47.351513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:47.358617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:47.359089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:47.359961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:47.360266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:47.391865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-42deacfc-b4c2-48ec-a534-43478da1991b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045412 Jul 20 15:19:47.392450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1132/1688] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:47.456899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:47.457475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:47.464650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:47.465104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:47.465866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:47.466302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:47.607268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:47.616055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-11b1c6bf-6e4f-42ad-9002-b6f6542730e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163190 Jul 20 15:19:47.616674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 557/1689] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:47 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:48.248871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:19:48.249252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:19:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:19:48.408281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:48.409785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:48.417468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:48.417921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:48.418675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:48.419136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:48.450499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-605b6e03-889d-499a-8135-06826da26cb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048918 Jul 20 15:19:48.451058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1133/1690] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:48.634908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:48.635659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:48.644014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:48.644539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:48.645468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:48.646218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:48.792743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:48.801150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5d373c6f-59f9-4612-a530-5c9f1515bafe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170991 Jul 20 15:19:48.801687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 558/1691] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:48 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:49.466646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:49.468214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:49.479498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:49.480242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:49.481727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:49.482264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:49.516316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c006a67-eedc-45d0-a3a4-eaa2664d5423 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056399 Jul 20 15:19:49.516810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1134/1692] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:49.820734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:49.821369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:49.828592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:49.829037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:49.829822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:49.830245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:49.964339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:49.972888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-013ffb04-a4b0-4671-8f99-304cff43341a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.156483 Jul 20 15:19:49.973436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 559/1693] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:50.528415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:50.529356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:50.536460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:50.536969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:50.537735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:50.538170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:50.568599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f89addd5-77e0-4bc2-af43-c6a4467c1b0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044183 Jul 20 15:19:50.569374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1135/1694] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:50.990760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:50.991441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:50.999032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:50.999504 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:51.000312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:51.000741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:51.019368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:51.133532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:51.142449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-afb40079-dd13-4a71-a04e-90cbbdf9b293 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.155725 Jul 20 15:19:51.143026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 560/1695] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:50 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:51.584688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:51.586009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:51.593158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:51.593624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:51.594397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:51.594833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:51.626202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9ba3930-89fe-43be-a443-c53fd2c8ec5f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048055 Jul 20 15:19:51.626771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1136/1696] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:52.163754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:52.164441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:52.172309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:52.172837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:52.173763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:52.174269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:52.323658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:52.331998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-11174531-eb66-4d20-ad3e-fd3a670974ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174608 Jul 20 15:19:52.332542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 561/1697] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:52 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:52.640954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:52.642372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:52.651094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:52.651576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:52.652339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:52.652772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:52.683990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d56378d0-7cdc-442b-88cb-61ee8ffba577 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048358 Jul 20 15:19:52.684468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1137/1698] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:53.353751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:53.355216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:53.366948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:53.367501 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:53.368370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:53.368809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:53.505856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:53.514412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-89bebe0d-a316-4396-a54b-85a442c955f1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167407 Jul 20 15:19:53.514994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 562/1699] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:53.698077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:53.699194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:53.708761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:53.709474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:53.710357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:53.710809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:53.742353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-079d7ed8-5640-4ddf-bc69-2ac57d3cca61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048887 Jul 20 15:19:53.742915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1138/1700] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:54.532518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:54.533136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:54.535566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=52 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:54.541511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:54.541958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:54.542735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:54.543208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:54.687411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:54.700430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4f31051a-e021-4bf1-ae78-fe657cd7e499 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.171616 Jul 20 15:19:54.701366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 563/1701] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:54 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:54.754247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:54.755140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:54.757531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=58 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:19:54.763090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:54.763579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:54.764349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:54.764786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:54.795385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c6be7576-121c-4d8a-8ef3-bf0ba6b5f506 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045184 Jul 20 15:19:54.795927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1139/1702] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:55.726134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:55.726783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:55.734705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:55.735231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:55.736079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:55.736545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:55.807706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:55.808653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:55.818140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:55.818782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:55.819718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:55.820146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:55.851579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8629282b-995d-4797-b3d3-a826003ce27f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047765 Jul 20 15:19:55.852157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1140/1703] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:55.884598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:55.893072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a160f511-fe3e-4858-9d70-6f5538095ac4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173672 Jul 20 15:19:55.893642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 564/1704] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:55 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:56.864222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:56.865179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:56.872783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:56.873222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:56.873969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:56.874397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:56.904361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53d58340-e0ed-4835-b69e-14671dd3362e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044083 Jul 20 15:19:56.904914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1141/1705] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:56 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:56.910194 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:56.910818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:56.918016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:56.918470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:56.919290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:56.919723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:57.054959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:57.063529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1e4b3b91-ba10-4c98-8c55-f2bce52003b7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.157171 Jul 20 15:19:57.064127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 565/1706] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:57.918277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:57.919271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:57.926349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:57.926796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:57.927616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:57.928048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:57.959094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b014057-e107-40bf-9899-079ab7342fba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044999 Jul 20 15:19:57.959824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1142/1707] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:58.084947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:58.085919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:58.097068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:58.097804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:58.099087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:58.099811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:58.255321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:58.263912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-81396062-e9cc-4990-b675-f024bf32e0dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.185558 Jul 20 15:19:58.264473 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 566/1708] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:58 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:19:58.975706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:58.977171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:58.984501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:58.984948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:58.985694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:58.986121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:59.017614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8ff9851f-aa95-401a-96b9-05ccd6311cff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048718 Jul 20 15:19:59.018197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1143/1709] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:19:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:19:59.281675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:19:59.282367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:19:59.290251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:59.290746 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:59.291671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:19:59.292130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:19:59.435396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:19:59.444074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd3d742b-4853-4f80-8ae7-260076448b46 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166585 Jul 20 15:19:59.444660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 567/1710] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:19:59 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:00.033614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:00.035113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:00.045660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:00.046405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:00.047729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:00.048445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:00.081072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab63397c-8cef-4186-8c47-671f111197eb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054587 Jul 20 15:20:00.081634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1144/1711] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:00.466206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:00.467227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:00.478816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:00.479321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:00.480104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:00.480530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:00.624280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:00.633130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5d51e643-269b-4468-973f-0397c6454201 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174071 Jul 20 15:20:00.633690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 568/1712] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:00.924490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:20:00.924851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:20:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:20:01.097264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:01.098792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:01.107351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:01.107801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:01.108565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:01.108998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:01.140519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c40bf174-197e-47d9-8f36-bb08d023383b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049819 Jul 20 15:20:01.141075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1145/1713] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:01.654410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:01.655464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:01.666431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:01.666885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:01.667682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:01.668118 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:01.814690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:01.823750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-68dba58e-0ccb-4b65-a9ef-785c28fee292 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176303 Jul 20 15:20:01.824312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 569/1714] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:01 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:02.157178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:02.158683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:02.168679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:02.169133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:02.169907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:02.170334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:02.201171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc157411-87ef-41b4-93a7-fe44afaff508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050868 Jul 20 15:20:02.201722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1146/1715] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:02 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:02.845004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:02.845950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:02.856739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:02.857205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:02.857966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:02.858399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:03.006233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:03.014578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-43ac2137-4665-4037-b954-9e24093742f9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176306 Jul 20 15:20:03.015181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 570/1716] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:02 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:03.216548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:03.217490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:03.225322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:03.225815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:03.226675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:03.227177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:03.255574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:03.260375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c839a681-fa82-4feb-8d3b-2e9537fbc542 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050067 Jul 20 15:20:03.260924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1147/1717] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:04.036365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:04.037290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:04.048600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:04.049357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:04.050618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:04.051401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:04.190819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:04.201775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-adb1a626-30cc-4220-a8da-29e9285a942f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172000 Jul 20 15:20:04.202325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 571/1718] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:04 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:04.277948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:04.279489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:04.290931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:04.291629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:04.292959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:04.293393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:04.322825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-711c5764-363a-418f-afdd-8213828f0f9b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051420 Jul 20 15:20:04.323410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1148/1719] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:05.219073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:05.219669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:05.226598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:05.227040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:05.227861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:05.228297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:05.334146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:05.335105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:05.342272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:05.342639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:05.343437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:05.343940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:05.367849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:05.375010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-813ebe75-715f-4a14-83fc-14071b71ba32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044946 Jul 20 15:20:05.378179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1149/1720] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:20:05.378359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a647f428-fd87-46e8-a1d4-64caa176dac0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163226 Jul 20 15:20:05.378724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 572/1721] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:05 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:06.390575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:06.392875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:06.398676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:06.399724 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:06.403477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:06.404044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:06.404718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:06.405147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:06.407244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:06.407692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:06.408453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:06.408886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:06.435368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5860ff54-75f9-4fe7-b8ae-5f415983cd0a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051212 Jul 20 15:20:06.435941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1150/1722] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:06.544022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:06.552580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d5417b53-fcbc-4a19-95d4-fddb570ff8ba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.160077 Jul 20 15:20:06.553152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 573/1723] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:06 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:07.447867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:07.448795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:07.455705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:07.456164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:07.456914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:07.457347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:07.488637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-33b1fff9-83a8-4a97-b630-a7a64ae59caf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044807 Jul 20 15:20:07.489234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1151/1724] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:07.575250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:07.575862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:07.582938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:07.583438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:07.584204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:07.584670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:07.797352 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:07.805999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1d8bb92e-54f4-4b8a-90fc-83a992b11134 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.234664 Jul 20 15:20:07.806589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 574/1725] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:07 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:08.501893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:08.502860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:08.509808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:08.510259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:08.511065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:08.511557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:08.540902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b03188d2-55a8-41a2-b4cf-bf2ea757c623 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042995 Jul 20 15:20:08.541449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1152/1726] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:08.778353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:20:08.779191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:20:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:20:08.823951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:08.824567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:08.831038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:08.831524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:08.832279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:08.832709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:08.973350 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:08.981791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a2d98471-d25f-44e3-8507-484de1c6d82c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161737 Jul 20 15:20:08.982325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 575/1727] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:08 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:09.557322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:09.558851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:09.569919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:09.570658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:09.571953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:09.572672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:09.603147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0df59c3-a509-405e-91e8-db307ee78af0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052795 Jul 20 15:20:09.603686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1153/1728] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:10.002130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:10.002901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:10.011977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:10.012569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:10.013582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:10.014160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:10.169506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:10.178183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e6fed4e8-41a9-4b14-8979-0c451ec58cd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.181192 Jul 20 15:20:10.178755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 576/1729] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:09 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:10.615276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:10.616235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:10.623708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:10.624158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:10.624884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:10.625381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:10.655805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44796837-554e-4b3f-9d53-20fbde2dd2ab tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044568 Jul 20 15:20:10.656337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1154/1730] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:11.199868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:11.200573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:11.208623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:11.209153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:11.210047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:11.210557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:11.215265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:11.353210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:11.365255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ba2835d7-b892-4b5a-85e6-d453584cd48e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.171672 Jul 20 15:20:11.366063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 577/1731] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:11 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:11.671969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:11.673459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:11.684463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:11.684906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:11.685660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:11.686093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:11.715471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebd6bd8d-7a93-4a91-8e64-755878f2c311 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050119 Jul 20 15:20:11.716012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1155/1732] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:12.388604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:12.389580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:12.400781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:12.401571 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:12.402879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:12.403347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:12.548662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:12.560550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a0eae9b6-c925-4c69-8660-d6ffdf86f0f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.178369 Jul 20 15:20:12.561116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 578/1733] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:12 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:12.732504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:12.733332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:12.741991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:12.742563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:12.743745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:12.744651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:12.780537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87c3890d-0abb-4411-bad6-874949941ece tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054013 Jul 20 15:20:12.781092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1156/1734] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:13.581949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:13.582993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:13.592651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:13.593095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:13.593843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:13.594264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:13.738154 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:13.749598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ad3dd282-7b3a-4fe1-8662-8dd3b5df0dba tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174226 Jul 20 15:20:13.750161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 579/1735] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:13 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:13.792371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:13.793320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:13.800529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:13.800976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:13.801721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:13.802153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:13.832499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b92c39ec-4e04-4f14-bbd8-866bcc7dc5c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044040 Jul 20 15:20:13.832973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1157/1736] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:14.771263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:14.772275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:14.783348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:14.784096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:14.785347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:14.786068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:14.844291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:14.845211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:14.851924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:14.852415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:14.853180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:14.853647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:14.884508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a06aa01d-f122-460f-b122-59edd7f60797 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044045 Jul 20 15:20:14.885056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1158/1737] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:14.921916 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:14.929945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b357fd30-99bf-443f-bbd8-6da23990b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165490 Jul 20 15:20:14.930560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 580/1738] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:14 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:15.900716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:15.902204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:15.909630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:15.910079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:15.910831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:15.911298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:15.941315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eb46ce41-ebf7-4e79-80a4-b0e59e4c3a90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047301 Jul 20 15:20:15.941871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1159/1739] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:15 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:15.950954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:15.951586 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:15.958444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:15.958887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:15.959671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:15.960098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:16.102051 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:16.114368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8e107a74-7299-49b6-9692-7f4ec1b20f53 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167119 Jul 20 15:20:16.115234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 581/1740] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:15 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:16.953803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:16.954713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:16.962238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:16.962680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:16.963461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:16.963885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:16.993544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57842022-2e66-4920-99ac-05e532d8e277 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043925 Jul 20 15:20:16.994398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1160/1741] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:17.138027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:17.138851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:17.146486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:17.146987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:17.147872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:17.148347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:17.291932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:17.300999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-008b3c92-7fec-46d1-a9b0-ffca237c0395 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169001 Jul 20 15:20:17.301577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 582/1742] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:17 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:18.006485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:18.007492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:18.014405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:18.014849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:18.015629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:18.016060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:18.045165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85319ccd-4e26-4597-b3fb-ff681312a321 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042870 Jul 20 15:20:18.045708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1161/1743] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:18.318593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:18.319219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:18.330803 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:18.331288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:18.332141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:18.332576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:18.473951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:18.482893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-25366e17-28b4-4a17-b065-90ca84463acb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168192 Jul 20 15:20:18.483516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 583/1744] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:18 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:19.057437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:19.058371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:19.065324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:19.065775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:19.066531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:19.066962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:19.097055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-75fd3698-12db-4ae4-ae32-e1e064643f43 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043675 Jul 20 15:20:19.097603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1162/1745] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:19.499751 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:19.500367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:19.507403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:19.507882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:19.508645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:19.509077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:19.641944 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:19.650338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-00cf3856-883d-446a-b8eb-d873c3708c4d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.154486 Jul 20 15:20:19.650891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 584/1746] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:19 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:20.113316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:20.114785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:20.125817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:20.126528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:20.127812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:20.128724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:20.161277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc4f8d10-d603-4555-a6be-53307bc57d6b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054517 Jul 20 15:20:20.161828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1163/1747] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:20.670455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:20.671068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:20.678365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:20.678815 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:20.679617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:20.680661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:20.826567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:20.835081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d625e9cd-2a6c-4541-a987-85132e60f5b5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170198 Jul 20 15:20:20.835723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 585/1748] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:20 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:21.177199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:21.178690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:21.188346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:21.188796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:21.189545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:21.189981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:21.207235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:21.221254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b0e9e69-5a07-41c7-9dc1-7b02f5d912f1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050647 Jul 20 15:20:21.221831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1164/1749] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:21 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:21.860462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:21.861518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:21.873305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:21.873843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:21.874621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:21.875143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:22.014607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:22.028068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f9e97138-8b69-408d-a470-d96029d45409 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.174001 Jul 20 15:20:22.028879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 586/1750] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:21 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:22.236484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:22.237566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:22.245895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:22.246425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:22.247362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:22.247881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:22.282507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce9f5954-aeab-451b-9dd9-21c85ff56d21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051751 Jul 20 15:20:22.283112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1165/1751] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:22 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:23.050103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:23.051378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:23.061996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:23.062701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:23.064016 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:23.064734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:23.208174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:23.216491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-81cb2ec5-1a9f-4b47-a233-7bc45de40ddf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173460 Jul 20 15:20:23.217080 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 587/1752] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:23 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:23.300444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:23.301843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:23.312215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:23.312666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:23.313410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:23.313844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:23.343360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e32128f1-0229-459a-b9d6-5f55bfcfd0fa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050095 Jul 20 15:20:23.343921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1166/1753] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:24.238353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:24.239352 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:24.250584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:24.251014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:24.251797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:24.252234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:24.355862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:24.356796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:24.363921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:24.364351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:24.365131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:24.365571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:24.396319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:24.404777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d4311626-dc8f-40a7-8f53-fc4a0463ef1d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173067 Jul 20 15:20:24.405601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 588/1754] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:24 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:24.415451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-14d434da-197e-4f75-9af9-afb1aad06dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063065 Jul 20 15:20:24.416501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1167/1755] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:25.426073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:25.427096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:25.430862 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=52 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:25.432845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:25.434335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:25.438341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=58 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:25.440090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:25.441056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:25.442251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:25.443011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:25.446844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:25.447925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:25.449139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:25.449849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:25.483711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-50268603-9dfa-4bdd-816a-cb3f5dc388d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057624 Jul 20 15:20:25.484311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1168/1756] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:25.604022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:25.612315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-926a7840-2b42-4338-b9e0-e1c90aac9a7f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.192959 Jul 20 15:20:25.613041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 589/1757] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:25 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:26.499550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:26.501088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:26.509373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:26.509824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:26.510594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:26.511026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:26.540607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-71233db4-481a-42bb-9246-f87813987a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048030 Jul 20 15:20:26.541167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1169/1758] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:26.630303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:26.630941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:26.637745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:26.638186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:26.638951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:26.639483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:26.790952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:26.800067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-71bdfd08-aa6c-412f-a562-b4d495bbaf94 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173695 Jul 20 15:20:26.800620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 590/1759] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:26 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:27.552085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:27.553058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:27.560061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:27.560511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:27.561263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:27.561699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:27.592777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-14716ff7-59ed-4059-af11-222b117fb9a1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044436 Jul 20 15:20:27.593463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1170/1760] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:27 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:27.822166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:27.823007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:27.833959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:27.834742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:27.836054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:27.836787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:27.973760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:27.985020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d1f3fac-55e4-42c3-8935-afc5732a6076 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169309 Jul 20 15:20:27.985803 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 591/1761] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:27 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:28.607089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:28.608417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:28.617722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:28.618407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:28.619605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:28.620208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:28.650756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3cc9ad83-e4b2-4a18-a6cb-cc3efcf5767e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049534 Jul 20 15:20:28.651351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1171/1762] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:29.012731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:29.013671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:29.021356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:29.021824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:29.022571 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:29.022994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:29.167578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:29.176931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7bf1fce6-9aae-48ac-bf7d-7414f6cb65d1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.171710 Jul 20 15:20:29.177540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 592/1763] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:29 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:29.666442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:29.668001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:29.676788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:29.677234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:29.677983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:29.678408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:29.708576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f81cd9f7-0e37-4db3-a613-f06cb719ddee tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048920 Jul 20 15:20:29.709183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1172/1764] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:30.197951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:30.198929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:30.210000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:30.210749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:30.212063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:30.212789 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:30.359478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:30.363341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:30.368824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4f5c250d-e2c3-4a14-ae89-3ec3bab6be32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.177181 Jul 20 15:20:30.369380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 593/1765] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:30 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:30.725229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:30.726786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:30.737206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:30.737953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:30.739240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:30.740060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:30.774148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c508e6c-7f00-40dd-ac07-e7e61cdf1d6a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056143 Jul 20 15:20:30.774694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1173/1766] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:31.386695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:31.387395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:31.396288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:31.396884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:31.398013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:31.398576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:31.546296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:31.559112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b8943f07-382c-4354-8a83-d7aa13d0fb92 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.176389 Jul 20 15:20:31.559879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 594/1767] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:31 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:31.785653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:31.786581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:31.793516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:31.793963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:31.794715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:31.795181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:31.825372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-19501602-fe19-4a76-97ac-de8b8fe91db2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043771 Jul 20 15:20:31.825969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1174/1768] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:31 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:32.582177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:32.582773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:32.589516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:32.589961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:32.590720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:32.591173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:32.728766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:32.739329 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4fbfa4e8-8c1e-4aee-93a5-0c9aee5b4d3f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162504 Jul 20 15:20:32.739900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 595/1769] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:32 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:32.841981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:32.843548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:32.854462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:32.855255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:32.856484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:32.857193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:32.888561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4468436f-dc22-4cd9-bbe4-cc0ee7c19a19 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053219 Jul 20 15:20:32.889104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1175/1770] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:32 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:33.760522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:33.761174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:33.772040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:33.772616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:33.773712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:33.774292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:33.900660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:33.901636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:33.909129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:33.909565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:33.910328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:33.910724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:33.913091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:33.921426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a8d09848-e2ed-413e-bcda-4f91fa29b6a9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167321 Jul 20 15:20:33.922242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 596/1771] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:33 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:33.941889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5a07e486-235e-4a1c-924b-dd25b8456713 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045218 Jul 20 15:20:33.942479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1176/1772] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:34.942763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:34.943809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:34.955144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:34.955641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:34.956229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:34.956660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:34.956816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:34.957825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:34.964506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:34.964944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:34.966017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:34.966455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:34.998096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f3d33be3-0f4f-4d1c-b07e-21c3b61b37ca tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046674 Jul 20 15:20:34.998706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1177/1773] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:34 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:35.089593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:35.101363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9edb7185-5160-46c8-98fe-e3b22d05bcc9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165212 Jul 20 15:20:35.102090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 597/1774] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:34 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:36.014315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:36.015872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:36.026834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:36.027646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:36.029043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:36.029753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:36.060780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b8ad6cc-a374-4527-b2eb-52b5f45214cc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053376 Jul 20 15:20:36.061315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1178/1775] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:36.123627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:36.124662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:36.132411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:36.132856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:36.133613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:36.134045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:36.272599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:36.281099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3156cfa6-0e2b-43c7-ba1b-08c8bca5d2e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164422 Jul 20 15:20:36.281655 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 598/1776] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:36 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:37.078466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:37.080038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:37.087466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:37.087929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:37.088708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:37.089158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:37.120568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7d300d2f-26bd-4532-b0e9-7a896a75cb9a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048995 Jul 20 15:20:37.121129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1179/1777] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:37 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:37.301441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:37.302027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:37.309470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:37.309831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:37.310838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:37.311285 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:37.456931 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:37.465184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ed112ce8-7846-4aad-a389-961b890d4a25 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169055 Jul 20 15:20:37.465777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 599/1778] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:37 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:38.136803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:38.138378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:38.149744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:38.150494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:38.151814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:38.152534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:38.185394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3577b19-0f75-40cc-a263-814fd67d55cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055752 Jul 20 15:20:38.185942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1180/1779] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:38.486259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:38.487363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:38.499298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:38.500043 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:38.501315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:38.502035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:38.636116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:38.644696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a1f2559a-6c6d-42dd-9f87-cebb6408e466 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165225 Jul 20 15:20:38.645274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 600/1780] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:38 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:39.201786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:39.203410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:39.214548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:39.215354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:39.216604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:39.217339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:39.223464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:39.250584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e55df16-41e3-4351-b1bf-bccf6f90777b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055687 Jul 20 15:20:39.251173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1181/1781] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:39.666343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:39.667444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:39.679411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:39.680209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:39.681524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:39.682238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:39.840642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:39.849071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1d5db152-a99a-472f-9219-00a68a28c6cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.189567 Jul 20 15:20:39.849620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 601/1782] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:39 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:40.266803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:40.268441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:40.276376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:40.276823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:40.277573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:40.278002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:40.309282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f67f6c4-7a55-4a31-bdef-1b5a7f8d9ada tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049318 Jul 20 15:20:40.309844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1182/1783] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:40 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:40.869851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:40.870687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:40.878895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:40.879525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:40.880475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:40.881018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:41.023456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:41.037231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2b390c60-26a0-4bb9-b87c-2038007f4ec1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.173516 Jul 20 15:20:41.038109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 602/1784] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:40 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:41.322514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:41.323615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:41.331772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:41.332280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:41.333151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:41.333635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:41.365605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-384b4c7d-9645-4aeb-a79d-0350509ed2f5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047803 Jul 20 15:20:41.366173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1183/1785] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:42.058840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:42.059824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:42.069971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:42.070412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:42.071205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:42.071634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:42.211428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:42.219675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b771108d-6ac4-474a-b29b-bb3eb7cd50e8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167099 Jul 20 15:20:42.220255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 603/1786] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:42.382293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:42.383716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:42.394880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:42.395677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:42.396942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:42.397774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:42.428863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-08166a20-3fe8-4bf7-a2e9-b0ae83e16b60 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053329 Jul 20 15:20:42.429449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1184/1787] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:42 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:43.241232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:43.242103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:43.249883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:43.250328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:43.251092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:43.251538 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:43.389214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:43.398130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4632b2ef-a477-46fe-8fe1-45744c1956e7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163577 Jul 20 15:20:43.398709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 604/1788] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:43 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:43.440517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:43.441442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:43.448432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:43.448881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:43.449629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:43.450057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:43.481254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2feb9ea3-a836-4721-8bf3-5d0df2b19fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044636 Jul 20 15:20:43.484623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1185/1789] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:43 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:44.425056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:44.426165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:44.434547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:44.434991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:44.435784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:44.436209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:44.493766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:44.494813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:44.501650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:44.502103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:44.502871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:44.503340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:44.534237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a22690b4-e740-4e4e-a588-2da519a07f4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044955 Jul 20 15:20:44.535113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1186/1790] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:44.575159 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:44.586646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4f6cda0d-2c73-4486-a0e2-7b1b17d76d87 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167846 Jul 20 15:20:44.587463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 605/1791] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:44 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:45.551411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:45.553105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:45.564623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:45.565367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:45.566624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:45.567394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:45.599374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b7a2d08c-42c2-41ee-aeb3-f15a9eeed0da tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055361 Jul 20 15:20:45.599958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1187/1792] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:45.604382 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:45.605049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:45.612762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:45.612762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:45.613576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:45.613695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:45.755491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:45.765330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-93616e9e-89df-4e13-8a86-7cec34dc1f71 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165017 Jul 20 15:20:45.765914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 606/1793] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:45 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:46.616963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:46.618380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:46.629626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:46.630173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:46.631030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:46.631491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:46.668067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8232edb1-4ca8-42cf-9a00-9f8c2c866349 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059826 Jul 20 15:20:46.668659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1188/1794] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:46 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:46.787039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:46.787968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:46.799230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:46.799892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:46.801173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:46.801898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:46.942105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:46.951152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b6239135-54f3-4e05-a760-dca891f902e1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170699 Jul 20 15:20:46.952006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 607/1795] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:46 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:47.680228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:47.681139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:47.687969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:47.688414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:47.689155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:47.689584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:47.718622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b717c31a-3a64-4f44-89b3-b9668d9d0e96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042336 Jul 20 15:20:47.719209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1189/1796] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:47.972718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:47.973573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:47.982757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:47.983295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:47.984180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:47.984752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:48.120386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:48.129850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b0778eb5-cd7c-442d-b66c-836cf93d73d8 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163736 Jul 20 15:20:48.130439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 608/1797] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:47 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:48.734580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:48.736551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:48.745057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:48.745513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:48.746432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:48.746946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:48.779517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d37a08f2-ae53-4b51-8a8d-1874a81f28f4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051251 Jul 20 15:20:48.780113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1190/1798] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:49.147900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:49.148498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:49.155633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:49.156106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:49.156903 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:49.157328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:49.295727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:49.304052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d7a4a09c-961c-4306-a2b8-cfa0a8e6675e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.160148 Jul 20 15:20:49.304618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 609/1799] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:49.791925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:49.792761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:49.799843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:49.800290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:49.801036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:49.801473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:49.831221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f94db548-df54-4d82-9aa7-b4b7bcd5a005 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043436 Jul 20 15:20:49.831783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1191/1800] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:50.326117 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:50.327142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:50.338328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:50.339113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:50.340190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:50.340638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:50.359291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:50.474766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:50.484094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-32532625-bf5d-43da-971a-333aad89ec7d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164621 Jul 20 15:20:50.484675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 610/1801] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:50 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:50.847322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:50.848810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:50.858421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:50.858870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:50.859661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:50.860085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:50.890455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-627eb202-730a-4cab-846b-e7d4e3afdff9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049924 Jul 20 15:20:50.891093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1192/1802] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:51.505665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:51.506307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:51.513813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:51.514264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:51.515068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:51.515507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:51.664207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:51.672469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4ac35c9f-db17-4b54-a5fc-3a2468ee6543 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170710 Jul 20 15:20:51.673035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 611/1803] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:51 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:51.905279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:51.906121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:51.912454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:51.912909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:51.913668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:51.914095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:51.944068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-03180e00-b5d6-4d33-8fc1-58ffacc46f03 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044935 Jul 20 15:20:51.944618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1193/1804] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:52.691183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:52.691896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:52.700129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:52.700685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:52.701618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:52.702151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:52.843600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:52.855153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-de08cabb-c656-4f94-856b-f8699ceeacbd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168684 Jul 20 15:20:52.855849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 612/1805] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:52 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:52.961525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:52.963093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:52.973981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:52.974722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:52.976106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:52.976836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:53.008212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0696c18e-8ccd-40ac-b4ef-1700819f3959 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053368 Jul 20 15:20:53.008753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1194/1806] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:53.877320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:53.878208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:53.887742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:53.888190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:53.888951 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:53.889380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.020060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:54.021019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:54.027186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:54.028022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.028520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.029291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.029725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.035512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3b0b6f8b-0007-4134-8c7a-3b2935aec46a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164958 Jul 20 15:20:54.036326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 613/1807] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:54.059970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ade9edb3-43b6-4f49-8bea-1140a283bd55 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043833 Jul 20 15:20:54.060531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1195/1808] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:20:54.705584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:54.706089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:54.712742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.713187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.713948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.714348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.749477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:54.750009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:54.751393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Going to try to terminate instance {{(pid=124397) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:20:54.756368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.758023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.758842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.759329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.786689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:20:54.828530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:54.834549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-30b64b85-73a3-4466-9db4-8c90c96af0c2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 204 len: 0 microversion: 2.1 time: 0.132635 Jul 20 15:20:54.835593 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 614/1809] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:20:54 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:20:54.845215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:54.845909 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:54.852418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.852740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.853620 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.854061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.881578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-acd90c12-192a-41ae-a180-33b4e70b66c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 204 len: 0 microversion: 2.20 time: 0.135951 Jul 20 15:20:54.882190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1196/1810] 10.222.0.15 () {60 vars in 1279 bytes} [Wed Jul 20 15:20:54 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 20 15:20:54.891173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:54.891884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:54.898302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.898771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.899587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:54.900008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:54.990600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:54.999308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-eedc5a07-c4a8-41b5-8f8b-7d8775617d95 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1726 microversion: 2.1 time: 0.159327 Jul 20 15:20:54.999913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 615/1811] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:54 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1726 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:20:55.052800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:55.053453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:55.059799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:55.060275 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:55.061114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:55.061559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:55.232224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:55.241400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:55.242794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:55.244366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8391aa5c-0398-46f3-8120-7ebb211ee965 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.357507 Jul 20 15:20:55.244897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1197/1812] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:20:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:20:55.249753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:55.250779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:55.253978 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:55.260895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:55.261348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:55.262158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:55.262612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:55.262776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d2501478-c83f-41ab-878a-08790d8e0dac tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.213779 Jul 20 15:20:55.263290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 616/1813] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:55 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:20:55.292582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-66a5ad55-929c-441c-b1b7-972d019d4b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046602 Jul 20 15:20:55.293138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1198/1814] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:56.020022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:56.021022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:56.025021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:56.034272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.035019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.036319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.037041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.265583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:56.266283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:56.268813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:56.273349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:56.275848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.276332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.277080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.277547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.283867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b78a08dd-60d8-4d0e-a5ca-7ff2373c2cda tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1726 microversion: 2.1 time: 0.270489 Jul 20 15:20:56.284460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 617/1815] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1726 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:56.289670 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:56.290331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:56.296784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.297228 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.299165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.299165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.832731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:56.841817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:56.843498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:56.844765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-94ddc0d8-52f0-4c91-8969-e9b10ea52fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.585345 Jul 20 15:20:56.845343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1199/1816] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:20:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:20:56.850011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:56.850946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:56.857356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.858556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.859374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:56.859814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:56.871650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:56.880003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-56b87959-0fcd-42e2-88fa-06ed14daea98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.594558 Jul 20 15:20:56.880869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 618/1817] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:56.897089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a8baf1ce-d043-4025-8772-e49e2dfdb1e0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050557 Jul 20 15:20:56.897622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1200/1818] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:56 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:57.312462 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:57.313035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:57.319985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.320472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.321238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.321673 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.397652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:57.406057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4f7f2baa-8c37-410b-b72e-8794a5dc52d4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1659 microversion: 2.1 time: 0.097826 Jul 20 15:20:57.406644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 619/1819] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:57 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1659 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:57.549490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-4ffc7861-7b23-4637-a6ac-a8f597a09502 req-6b7a5341-59cd-477a-afb7-1eeb44a43895 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6b566c6c-a418-4384-b99e-c7295e754e92", "name": "network-vif-deleted", "tag": "67324695-43bc-444a-9a35-3a3bbb87de0b"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:20:57.558529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4ffc7861-7b23-4637-a6ac-a8f597a09502 req-6b7a5341-59cd-477a-afb7-1eeb44a43895 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.559318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-4ffc7861-7b23-4637-a6ac-a8f597a09502 req-6b7a5341-59cd-477a-afb7-1eeb44a43895 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.573677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-4ffc7861-7b23-4637-a6ac-a8f597a09502 req-6b7a5341-59cd-477a-afb7-1eeb44a43895 service nova] Creating event network-vif-deleted:67324695-43bc-444a-9a35-3a3bbb87de0b for instance 6b566c6c-a418-4384-b99e-c7295e754e92 on hv-cinder-84902 Jul 20 15:20:57.579487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-4ffc7861-7b23-4637-a6ac-a8f597a09502 req-6b7a5341-59cd-477a-afb7-1eeb44a43895 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116824 Jul 20 15:20:57.580735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1201/1820] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:20:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:57.870298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:57.870298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:57.871291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.871756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.872725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.872990 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.898483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:57.899453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:57.908499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.909171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:57.910124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:57.910739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.065272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.074214 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.076232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.076943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dcc5b6f6-6ad6-45c1-9f4e-0aacc94d6e1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.217622 Jul 20 15:20:58.079236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 620/1821] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:20:57 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:20:58.082708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:58.083799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:58.089998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.090534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.091338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.091777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.094364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.102510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4e42435-16a8-44d9-8cd3-b54f43c686cc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.208696 Jul 20 15:20:58.103111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1202/1822] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:57 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:20:58.107967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:58.108463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:58.114409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.114799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.115595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.116030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.132043 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-947de2b8-1e3b-4857-872e-c6850bef9fef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052953 Jul 20 15:20:58.132933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 621/1823] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:58.142155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:20:58.221961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b104800a-5526-4ed6-bd82-d8c136794fe5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 204 len: 0 microversion: 2.1 time: 0.117102 Jul 20 15:20:58.222811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1203/1824] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:20:58 2022] DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:20:58.231792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:58.232719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:58.239251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.240031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.240943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.241384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.374814 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.383919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c70e6419-b5a7-4dfd-ac10-a49b7a199b73 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1721 microversion: 2.1 time: 0.156612 Jul 20 15:20:58.384610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 622/1825] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1721 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:58.424554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:58.425234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:58.431683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.432163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.432930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:58.433332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:58.501496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:58.509589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-927db916-835d-4987-96b9-728e1d66788e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089370 Jul 20 15:20:58.510157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1204/1826] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:58 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:59.097188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:59.097946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:59.105962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.106441 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.107224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.107688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.119866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:59.120483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:59.126800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.127288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.128057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.128492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.302780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.310031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.311876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.313433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.314722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-edec0a50-a63a-441f-8d6c-a5f371664329 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.222578 Jul 20 15:20:59.315306 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 623/1827] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:20:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:20:59.318454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c54622c-e691-440c-b456-943a12741eb0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.202744 Jul 20 15:20:59.328131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1205/1828] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:59 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:20:59.328321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:59.328321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:59.333486 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.334135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.334757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.335359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.365065 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b7c6ddfb-bc7f-4ee6-8f5d-7c88d096f75a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048719 Jul 20 15:20:59.365663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 624/1829] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:20:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:20:59.401569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:59.402179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:59.408800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.409234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.409992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.410743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.525641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:20:59.526319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:20:59.532529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.532974 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.533738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:20:59.534166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:20:59.552079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:20:59.607354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.615440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-88441961-ac84-4743-93ab-18d232f459e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.093504 Jul 20 15:20:59.616007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 625/1830] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:20:59.646677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:20:59.655305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74bc3cae-5fe4-4250-939c-c4a8ae9d4504 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1721 microversion: 2.1 time: 0.257514 Jul 20 15:20:59.655857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1206/1831] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:20:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1721 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:00.293388 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [req-d4f34cf0-5324-4df8-80d2-096b08de3311 req-35999f3a-53b4-4a65-8e65-2efc475c1eb5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52220445-7cc1-4635-9566-c6fd58b1fc96", "name": "network-vif-deleted", "tag": "715e7be7-dd5f-443a-b42e-d143d247ea7d"}]} {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:21:00.304249 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-d4f34cf0-5324-4df8-80d2-096b08de3311 req-35999f3a-53b4-4a65-8e65-2efc475c1eb5 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.304699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [req-d4f34cf0-5324-4df8-80d2-096b08de3311 req-35999f3a-53b4-4a65-8e65-2efc475c1eb5 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.316674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.compute.server_external_events [req-d4f34cf0-5324-4df8-80d2-096b08de3311 req-35999f3a-53b4-4a65-8e65-2efc475c1eb5 service nova] Creating event network-vif-deleted:715e7be7-dd5f-443a-b42e-d143d247ea7d for instance 52220445-7cc1-4635-9566-c6fd58b1fc96 on hv-cinder-84902 Jul 20 15:21:00.322100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [req-d4f34cf0-5324-4df8-80d2-096b08de3311 req-35999f3a-53b4-4a65-8e65-2efc475c1eb5 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032435 Jul 20 15:21:00.322961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 626/1832] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:21:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:00.334707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:00.335333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:00.336683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:00.337394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:00.344460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.345151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.345419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.346419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.346870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.348025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.349631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.350537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.544870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.553463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bf1c3cbe-7065-4246-90cd-0dfc7e2f057f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.221539 Jul 20 15:21:00.563107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 627/1833] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:21:00.563107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:00.563107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:00.566411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.566921 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.567795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.568329 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.572302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.583028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.584881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.586305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-716232c1-ebda-41b0-9849-e97704cd901d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.257080 Jul 20 15:21:00.586991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1207/1834] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:21:00.599173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8bc589de-8952-41b5-ad80-e07e42cbc611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043751 Jul 20 15:21:00.599765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 628/1835] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:00.632555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:00.633401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:00.640402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.641038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.641833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.642266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.672895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:00.674045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:00.680742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.681273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.682140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:00.682648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:00.708793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.717399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d842fdf-1439-4139-be3a-83d89010ac5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088710 Jul 20 15:21:00.726960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1208/1836] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:00.746682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:00.754914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-da1107de-cd93-49ef-83a6-bbdb5ad35671 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085985 Jul 20 15:21:00.755603 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 629/1837] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:01.575500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:01.576571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:01.588495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.589243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.590516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.591285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.604188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:01.604916 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:01.612294 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.612755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.613562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.613999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.799476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.806831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.808325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34fec85f-5cc0-4c82-828f-61600cefe450 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.239543 Jul 20 15:21:01.808909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1209/1838] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:01 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:01.814506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:01.815517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:01.816772 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.818490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.820036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c217ce54-9587-4647-9167-c353de317fa1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.219742 Jul 20 15:21:01.820742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 630/1839] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:01.822745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.823480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.824497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.825150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.825798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:01.826427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:01.833907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.834358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.835226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.835745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.856086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-63041cc1-a334-4de8-8c1b-ef3d9034e73b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046141 Jul 20 15:21:01.857578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1210/1840] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:01.862936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:01.863570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:01.870010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.870455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.871373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:01.871805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:01.904007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.912584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-101c21bb-b81b-46be-9cfa-6b82c0d83b55 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.090647 Jul 20 15:21:01.913232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 631/1841] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:01 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:01.934954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:01.938646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:01.944254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f254a2b-2516-457c-95c7-dc318138c725 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085715 Jul 20 15:21:01.944833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1211/1842] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:02.829071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:02.829773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:02.838407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:02.838970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:02.839914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:02.840138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:02.840646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:02.840858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:02.849607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:02.850179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:02.851081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:02.851600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.044265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.047189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.052776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-98074990-30fb-4162-a0c7-ead51e9d00ce tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.229568 Jul 20 15:21:03.053392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 632/1843] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:02 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:03.056576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.058411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.058646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:03.059715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:03.059886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-95639882-bfc2-4beb-8db9-cea4b204f995 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.225661 Jul 20 15:21:03.060449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1212/1844] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:03.071101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.071101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.071101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.071101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.077002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:03.077628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:03.084242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.084624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.085404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.085836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.098463 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-35d3798c-57b4-4bda-ac98-c54d68617ffa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043784 Jul 20 15:21:03.099186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 633/1845] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:03.103822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:03.104432 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:03.110609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.111086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.111856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:03.112323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:03.165933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.174377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce468c75-78c9-409d-af1c-fe9995dfc8c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.102720 Jul 20 15:21:03.174929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1213/1846] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:03.183444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:03.192423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5d4280fa-7731-4783-a0ba-a683b12fd91a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092186 Jul 20 15:21:03.193001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 634/1847] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:04.083275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:04.084337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:04.085697 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:04.086617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:04.094471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.094927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.095788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.095933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.096222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.096396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.097139 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.097560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.293465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.296752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.302659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.304210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.305384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30497589-113f-457e-90c5-3ee26fc62c17 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.228897 Jul 20 15:21:04.305530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1b5574e2-96a5-47bc-99b4-6062bf6e81fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.226332 Jul 20 15:21:04.305959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1214/1848] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:04.306339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 635/1849] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:04 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:04.310823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:04.311679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:04.311992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:04.313031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:04.319682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.320132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.320884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.321305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.324640 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.325060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.326083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.326558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.352252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7e254baa-305f-4d5e-90a9-7b5966b752db tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044448 Jul 20 15:21:04.352824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1215/1850] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:04.357352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:04.358261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:04.364417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.364851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.365665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:04.366092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:04.395923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.403928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bcd3a8ed-61e7-43bd-9591-02bdccb191f9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096622 Jul 20 15:21:04.404499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 636/1851] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:04.428785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:04.437018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bbbf6b60-b92f-45f5-a0d2-1ad9789a6a59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083286 Jul 20 15:21:04.437603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1216/1852] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:05.323565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:05.324091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:05.324460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:05.324547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:05.330742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.331250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.331448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.331893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.332013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.332461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.332699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.333081 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.555488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.559682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.565437 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-14cf9f71-8838-42d3-89dc-d20fb94254ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.246104 Jul 20 15:21:05.566130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 637/1853] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:05 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:05.569407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.570880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:05.571032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.571948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:05.572133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-99ae1a4d-19e3-4e74-912b-973b29960b8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.252497 Jul 20 15:21:05.572887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1217/1854] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:05.577600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:05.578139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:05.578449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.578964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.579818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.580361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.584100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.584557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.585314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.585753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.610301 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b3f58ddb-2d4b-432e-8464-bae5c05c426d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043058 Jul 20 15:21:05.610888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 638/1855] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:05.615280 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:05.615871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:05.622232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.622672 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.623560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:05.623995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:05.651722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.659977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bebdb68-555d-4ac4-a61d-152d7865683c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085899 Jul 20 15:21:05.660531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1218/1856] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:05.692569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:05.700812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c630e4d5-37dc-41b3-bbe5-159696095df5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088931 Jul 20 15:21:05.701547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 639/1857] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:06.587398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:06.588359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:06.595216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:06.596218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:06.599182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.599939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.601194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.601911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.606811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.607630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.608629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.608988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.802136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.806282 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.811027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72974695-d38a-4f2c-a900-ecbb5f8fa8f5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.229910 Jul 20 15:21:06.811720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1219/1858] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:06 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:06.816385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:06.817408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:06.818121 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.819938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.821272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-64e291c6-75f3-45c4-9e76-8b77bfcb0918 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.232843 Jul 20 15:21:06.821920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 640/1859] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:06.831370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.831370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.831370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.831370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.831758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:06.831758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:06.833720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.834252 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.835100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.835599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.854894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fb6a6487-4f21-40aa-b127-4a3301bf2785 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042219 Jul 20 15:21:06.855608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1220/1860] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:06.859912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:06.860502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:06.866244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.866717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.867486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:06.868221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:06.897079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.905439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-81d22c43-408d-413d-a322-0e914e888e61 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082414 Jul 20 15:21:06.906021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 641/1861] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:06.932626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:06.941040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2371a183-b564-4547-ba33-4d33d261fe76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084482 Jul 20 15:21:06.941602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1221/1862] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:06 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:07.832196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:07.833246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:07.843601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:07.843996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:07.844434 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:07.844621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:07.844965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:07.845422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:07.851540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:07.851968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:07.852718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:07.853148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.039073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.042134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.042607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:08.048556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-37aa29ec-ca62-491b-a53e-e20334075024 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.222940 Jul 20 15:21:08.049131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 642/1863] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:07 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:08.051466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.052958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.053632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:08.054138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d652b740-59a6-48d0-8f1f-b7c0f2362013 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.216307 Jul 20 15:21:08.054598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:08.054979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1222/1864] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:21:08.059265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:08.059868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:08.064120 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.064570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.065333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.065759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.066152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.066639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.067433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.067864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.095278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8aa44df5-3bf0-498d-9db0-f73bb6553269 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044970 Jul 20 15:21:08.096104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 643/1865] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:08.102602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:08.103254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:08.109522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.109967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.110753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:08.111227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:08.129500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.138153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a728121b-a83b-44f5-b19b-d9f1dcd8d8d5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082277 Jul 20 15:21:08.138794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1223/1866] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:08 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:08.179522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:08.187810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-41f96e9a-fe23-4bd1-8528-a31050836b24 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090723 Jul 20 15:21:08.188377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 644/1867] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:09.065703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:09.066296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:09.072115 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:09.072853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:09.073094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.073543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.074301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.074749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.079888 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.080405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.081132 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.081584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.271939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.276176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.283013 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.284427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.285578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d1a80426-ceed-44c5-b390-2511e924f1d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.217340 Jul 20 15:21:09.285700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-def2d611-5001-4f7e-b36f-833fe43eee41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.223834 Jul 20 15:21:09.286143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 645/1868] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:09.286296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1224/1869] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:09 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:09.291003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:09.291812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:09.291927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:09.293499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:09.298536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.298998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.299838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.300284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.300761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.301212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.301989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.302428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.331509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f9eafee-6e2c-4fe4-b37a-50c461e07ee0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044093 Jul 20 15:21:09.332058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1225/1870] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:09.336808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:09.337418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:09.344702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.344702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.344702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:09.345123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:09.364303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.372822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-55b48812-9696-4995-98a7-5c50ca178105 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085246 Jul 20 15:21:09.373359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 646/1871] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:09 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:09.409132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:09.417357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-67af2271-8d9a-4158-b1ae-181b2b1f71c0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084034 Jul 20 15:21:09.417927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1226/1872] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:10.306146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:10.307076 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:10.308050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:10.308883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:10.316347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.316970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.318073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.318457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.318656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.319103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.320180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.320789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.518087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.520197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.526641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-204a03c2-fac6-44e9-9b53-b6cea69f9bd9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.227304 Jul 20 15:21:10.527253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 647/1873] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:10 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:10.529030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.530703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.531911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d894891a-c553-4ddd-b6bd-b8578be87a58 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.230047 Jul 20 15:21:10.532072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:10.542645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1227/1874] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:10.542894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:10.543009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:10.543142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.543269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:10.543376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.543376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.543376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.546928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.547411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.548187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.548618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.567449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:10.572784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7b7501b8-ce3c-46c3-a2cf-8081d6539016 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044460 Jul 20 15:21:10.573375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 648/1875] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:10.578136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:10.578765 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:10.585072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.585521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.586284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:10.586717 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:10.612335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.620940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e1178b2f-982a-4272-9281-5c2ee5361a49 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087537 Jul 20 15:21:10.621506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1228/1876] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:10 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:10.647833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:10.656386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e57a822a-4b19-41a0-9d3c-f1baaf5e4f66 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082000 Jul 20 15:21:10.656924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 649/1877] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:11.547794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:11.548798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:11.554053 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:11.555186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:11.559954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.560798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.561542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.561974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.564142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.564587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.565344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.565762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.773788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.776917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.782242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86005dba-8c58-4e49-84f0-9ffdaf96b1fa tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.240808 Jul 20 15:21:11.782897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1229/1878] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:11 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:11.785972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.787939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:11.788224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.788897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:11.789660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a49fc941-9d2e-4d55-a170-bae5d6f6203c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.242261 Jul 20 15:21:11.790199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 650/1879] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:11.800372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.800372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.800372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.800372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.803774 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:11.804391 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:11.810291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.810900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.811694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.812119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.825990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-13295614-ac59-426c-878d-2d139ff96748 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041861 Jul 20 15:21:11.829981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1230/1880] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:11.831450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:11.832041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:11.838027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.838477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.839286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:11.839722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:11.878014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.886093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-51363e5e-fd65-4212-a3f4-b7158c396a2f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085874 Jul 20 15:21:11.886635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 651/1881] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:11 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:11.905111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:11.913563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-099669b3-a571-44b7-8bfd-f383ac449461 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085577 Jul 20 15:21:11.914217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1231/1882] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:12.803439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:12.805532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:12.811568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:12.812201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:12.814925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:12.815402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:12.816161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:12.816591 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:12.819830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:12.820291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:12.821054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:12.821482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.269058 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.275778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.277211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd102fe8-0b41-4013-985b-780cbcf3b8b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.480153 Jul 20 15:21:13.277878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 652/1883] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:12 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:13.284647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.284916 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:13.285997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:13.286210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.287443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a91d51b-83da-42c1-97c1-5a8c800ec565 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.482081 Jul 20 15:21:13.288758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1232/1884] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:13.292308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.292873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.293018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:13.293524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:13.293925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.294455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.302962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.303445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.304207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.304641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.323706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-026dba2d-184c-462c-949b-31c8c69cfe2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043996 Jul 20 15:21:13.324315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 653/1885] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:13.328691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:13.329303 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:13.335417 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.336147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.336624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:13.337054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:13.372843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.381205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-281475c4-0d80-498c-95df-f520fc5fdeb4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.091809 Jul 20 15:21:13.381921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1233/1886] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:13.400381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:13.408321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b5d934f9-54b9-4959-9d11-888f5dcd75bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083101 Jul 20 15:21:13.408892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 654/1887] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:14.298328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:14.299481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:14.309781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:14.310820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.310987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:14.311325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.312093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.312531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.317471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.317923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.318738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.319291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.506044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.509323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.514610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2616661e-f963-46b4-a02b-fb8b1162796c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.222918 Jul 20 15:21:14.515219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1234/1888] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:14 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:14.518346 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.519764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:14.519919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.520759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:14.521006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8f8db243-f5a5-45c1-a42d-2189ab73b41d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.217850 Jul 20 15:21:14.523292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 655/1889] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:14.526783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.527232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:14.527374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.527837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:14.527978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.528440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.533913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.534386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.535123 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.535556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.559532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bdbda831-8b0d-4aa7-9cd5-8d83f7682ab4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043144 Jul 20 15:21:14.560029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1235/1890] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:14.564432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:14.565036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:14.570952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.571430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.572178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:14.572603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:14.600194 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.609031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-40ab5ce1-471f-4ac3-b6ea-607f18ae9033 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086005 Jul 20 15:21:14.609583 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 656/1891] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:14.634243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:14.642659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-916b0fdc-419d-48a4-aa5c-117aa8d93877 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081600 Jul 20 15:21:14.643228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1236/1892] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:15.532070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:15.532677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:15.539336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:15.539820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.539935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:15.540036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.540678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.541103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.546929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.547415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.548176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.548610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.734550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.737688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.743156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d943c758-b0ee-4ef0-a768-721f47cdd075 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.214913 Jul 20 15:21:15.743801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 657/1893] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:15 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:15.746698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.748365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.748828 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:15.749616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2accdc4-998f-4dea-b35f-cd572d19686a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.214311 Jul 20 15:21:15.749965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:15.753503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1237/1894] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:15.754745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:15.755373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:15.757071 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.757508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.758307 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.758742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.761720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.762170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.762928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.763399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.787343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-56c8416d-95a6-4bcb-8e17-3f93fe8b7ccf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042377 Jul 20 15:21:15.787895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 658/1895] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:15 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:15.792360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:15.793576 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:15.799721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.800177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.801238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:15.801358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:15.819988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:15.826100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.834228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-43c7ee11-cb17-445c-9242-d75d8a4c4ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083028 Jul 20 15:21:15.834796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1238/1896] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:15.863645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:15.871971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-21750ca2-2300-4ad5-8cf1-244322bad520 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083115 Jul 20 15:21:15.872561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 659/1897] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:15 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:16.765251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:16.766092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:16.772099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:16.773118 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:16.777007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:16.777759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:16.779122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:16.779677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:16.781936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:16.782389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:16.783174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:16.783631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:16.977105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:16.980525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:16.991119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c255c0dd-b4f7-4fe8-85e8-576ceff47212 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.230394 Jul 20 15:21:16.991119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1239/1898] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:16 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:16.991426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:16.991426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:16.992934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-46ae88fe-821e-42a7-bd02-b8d7e983dad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.227151 Jul 20 15:21:16.993743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 660/1899] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:16.994555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:16.995561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:17.001831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.002287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.003070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.003514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.006075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:17.006690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:17.013026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.013543 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.014299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.014725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.050857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56c0b5c4-7ee9-41ac-a4b6-9090e52dbd96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060075 Jul 20 15:21:17.051471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1240/1900] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:17.056010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:17.056616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:17.062650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.063163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.063914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:17.064344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:17.082465 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:17.090621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7e8ffe20-c433-4a0f-9f7c-f9d358b7c590 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092884 Jul 20 15:21:17.091209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 661/1901] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:17.125961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:17.134488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9b3ad840-50dc-4b81-aa70-344dc1893344 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082081 Jul 20 15:21:17.135177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1241/1902] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:18.011912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:18.013156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:18.018637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:18.019737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:18.024208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.024953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.026187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.026940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.030487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.030972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.031750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.032202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.219718 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.223023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.227949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-36a43b6e-2bf3-43f6-91d1-7c604cfc2abf tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.223396 Jul 20 15:21:18.228607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 662/1903] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:18 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:21:18.232098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.233609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.234774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5f57f8b3-d8e2-4d22-b3b3-d705c27fe9c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.222574 Jul 20 15:21:18.235374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1242/1904] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:18.239856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:18.240457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:18.240915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:18.241962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:18.246667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.247145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.247972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.248419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.248594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.249040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.249793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.250226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.280017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1257b9dd-6dce-476a-bb68-be1388c98824 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043128 Jul 20 15:21:18.280528 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 663/1905] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:18.285032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:18.285635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:18.291768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.292210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.292960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:18.293377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:18.309733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.317850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-938baa0e-3ad2-4d27-a055-b91974a7b495 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081388 Jul 20 15:21:18.318640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1243/1906] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:18 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:18.354954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:18.364247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1ce9e2d8-0033-43ef-955f-6de2e8e6fb19 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082738 Jul 20 15:21:18.364825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 664/1907] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:19.249473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:19.250457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:19.257480 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:19.258540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:19.261927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.262683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.264006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.264443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.267235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.267688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.268473 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.268912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.461911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.464958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.470266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e95442e-6025-4c74-bcec-ea8a189687ff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.227720 Jul 20 15:21:19.471277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1244/1908] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:19 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:19.474331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.475748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.476005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:19.477011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:19.478056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-71a61548-565a-4994-a660-91a27bcdc120 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.227486 Jul 20 15:21:19.485314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.485314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.485314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.485314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.486165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 665/1909] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:19.490045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:19.491607 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:19.492705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:19.498908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.499387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.500142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.500587 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.516429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-98e27ee0-f4b9-4d99-b14b-cc8bcaa0339a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044043 Jul 20 15:21:19.517132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1245/1910] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:19.521528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:19.522130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:19.523810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:19.529332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.529778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.530528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:19.530954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:19.567802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.576021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-36d39f70-dfe8-4208-b48a-c550f4f8a904 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088522 Jul 20 15:21:19.576567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 666/1911] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:19.595928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:19.608225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53dbc99f-6b70-4abf-96a6-0f42568a4030 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090065 Jul 20 15:21:19.608999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1246/1912] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:19 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:20.492713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:20.494334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:20.497962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:20.500903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:20.501569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:20.504205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.504658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.505437 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.505881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.508247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.508701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.509462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.509899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.693596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.702258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-af12bc2d-6163-471f-8a23-9cf3fbfafb30 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.216400 Jul 20 15:21:20.702835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 667/1913] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:20 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:20.707371 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:20.708375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:20.709442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.714702 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.715179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.715941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.716410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.718202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.719894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.721063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b345e3c-e17f-4002-ae41-aff77b5bdb72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.225824 Jul 20 15:21:20.721656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1247/1914] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:20.726056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:20.726657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:20.732977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.733425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.734177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.734608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.747251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d9210a3b-6d0f-40e2-8d86-70d5272cc128 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043248 Jul 20 15:21:20.747804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 668/1915] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:20.752149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:20.752779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:20.758661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.759150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.759913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:20.760315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:20.796240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.805004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7b94b523-ba9a-4994-9c0f-271cd83f9015 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082383 Jul 20 15:21:20.805623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1248/1916] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:20 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:20.825209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:20.833311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0e26ed4b-5fba-4330-9fbe-c01bf1684f94 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084580 Jul 20 15:21:20.833881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 669/1917] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:21.723378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:21.724583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:21.731991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.732489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.733287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.733783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.739617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:21.740266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:21.749730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.750412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.752590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.752590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.954747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:21.958908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:21.964184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:21.965683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:21.966895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1d5c778-f4d5-436c-ada7-4bcca6143606 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.250966 Jul 20 15:21:21.967024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-53e0d570-b90c-4bb0-ab60-4cc613955090 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.231202 Jul 20 15:21:21.967700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1249/1918] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:21 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:21.967840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 670/1919] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:21.971868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:21.972456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:21.978564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.979023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.979777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.980268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.982619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:21.983644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:21.989632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.990082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:21.990830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:21.991288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.024609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef56542d-e93d-473b-87ac-361ee6611d0e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045920 Jul 20 15:21:22.025239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1250/1920] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:21 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:22.030417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:22.031022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:22.037008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.037474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.038369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.038798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.044127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:22.052445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-688c9a39-091b-403f-8933-97ee0fe4b096 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083984 Jul 20 15:21:22.053046 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 671/1921] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:22.102491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:22.110843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80866bc8-7388-4196-8c12-d6e97c2f46b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084641 Jul 20 15:21:22.111440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1251/1922] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:22.986511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:22.987172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:22.987570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:22.987684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:22.994246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.994711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.994935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.995449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.995627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.996114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:22.996268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:22.996692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.193901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.197506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.202455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7abbd3cb-f5be-4e9b-ab97-be11c16ebf03 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.220508 Jul 20 15:21:23.203195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 672/1923] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:22 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:23.207860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.208068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:23.209079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:23.209455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.210861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fdc121cd-f82c-40eb-a97c-e222cf214ee8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.227871 Jul 20 15:21:23.211648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1252/1924] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:23.216400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:23.217087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:23.219418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.219871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.221084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.221511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.223535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.224053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.224871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.225371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.251215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-61d396a3-370e-4615-a6f8-ac780b6e8fea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046772 Jul 20 15:21:23.251643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 673/1925] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:23.259224 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:23.259819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:23.265901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.266339 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.267106 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:23.267537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:23.293340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.301895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02c9d3bc-9ee8-4f4c-971c-6ed6f36365b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089332 Jul 20 15:21:23.302462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1253/1926] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:23.329316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:23.337476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-79ab0ae6-8fc2-41b1-a309-d5a1466494a5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081667 Jul 20 15:21:23.338069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 674/1927] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:24.224385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:24.225608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:24.233459 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:24.234492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:24.236482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.237222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.238483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.239260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.245648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.247239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.248407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.248849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.437642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.440708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.446114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0fb6bb96-c466-4d87-af16-979f7657536a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.228315 Jul 20 15:21:24.446660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1254/1928] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:24 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:24.449622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.451245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.451458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:24.452380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:24.452495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd09d2a9-5017-4b27-9922-6532933b466f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.225756 Jul 20 15:21:24.453133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 675/1929] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:24.457290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:24.457887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:24.458403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.458853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.459762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.460516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.463709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.464380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.465141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.465569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.470589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:24.490261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-18805c92-7a1d-4850-a139-8812a0213403 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042522 Jul 20 15:21:24.491179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1255/1930] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:24.495966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:24.496567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:24.502545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.502987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.503812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:24.504239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:24.529652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.537643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f5b3101f-80d6-441b-bf0f-bf4a48380cec tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083744 Jul 20 15:21:24.538181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 676/1931] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:24.569022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:24.579388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a43df5aa-8476-421b-88fd-08efb8351fd6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087204 Jul 20 15:21:24.579922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1256/1932] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:25.473151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:25.473756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:25.474672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:25.475360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:25.481949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.482993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.483326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.483606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.484122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.484265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.484551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.484711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.676375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.679364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.685237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.686700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.687545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3ec40bc-98f7-4853-8cce-aeff6227d7a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.216847 Jul 20 15:21:25.687927 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d1755353-0767-49b6-b7bb-40150d09e419 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.219485 Jul 20 15:21:25.688427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1257/1933] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:25 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:25.688853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 677/1934] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:25.693380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:25.693964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:25.700110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.700569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.701330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.701766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.703603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:25.704547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:25.710457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.710892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.711716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.712149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.741180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e4d35da0-1dfd-4f05-a271-9803b545a0e6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041792 Jul 20 15:21:25.741799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1258/1935] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:25.746763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:25.747390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:25.753599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.754047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.754861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:25.755377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:25.765235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.773723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8a2635a5-6195-4681-80c3-bab9eee30dd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083832 Jul 20 15:21:25.774328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 678/1936] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:25.823230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:25.831612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-974482a0-962e-421f-b129-2476f0240543 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088489 Jul 20 15:21:25.832181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1259/1937] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:26.705310 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:26.705902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:26.706293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:26.706418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:26.712889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.713048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.713336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.713522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.714144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.714292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.714576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.714723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.910251 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:26.913452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:26.918623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ee5ee6cd-e6aa-4d12-8b57-53e7cd3f0c01 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.217145 Jul 20 15:21:26.919288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 679/1938] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:26 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:26.922482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:26.923997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:26.924138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:26.925150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5ee75062-4684-4fb8-8a61-17e4957c8b70 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.223308 Jul 20 15:21:26.925276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:26.928675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1260/1939] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:26.930378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:26.930979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:26.931611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.932127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.932933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.933454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.937732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.938156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.939265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.939699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.963355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a228f29a-8c8b-4575-a1a5-e5e1acbad6c1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042812 Jul 20 15:21:26.963964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 680/1940] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:26.968520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:26.969166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:26.975213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.975663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:26.976428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:26.976861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:27.001594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:27.004993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:27.010421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-41f047bf-3981-41e7-8999-f4aeb36f707e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083630 Jul 20 15:21:27.010994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1261/1941] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:27.040893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:27.049671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-036e9a99-9c09-48e0-9262-429a4972731b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084770 Jul 20 15:21:27.050245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 681/1942] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:27.939853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:27.940709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:27.946066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:27.946906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:27.950160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:27.950758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:27.951834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:27.952413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:27.956477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:27.957073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:27.958103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:27.958683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.154418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.157648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.165107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-93593247-8a21-48cb-806c-c0f3c84a8544 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.231806 Jul 20 15:21:28.166169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1262/1943] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:27 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:28.169689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.171341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:28.171469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.172315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:28.172734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fc00c8df-422a-43c5-914c-4659ad825e02 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.231977 Jul 20 15:21:28.183549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 682/1944] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:28.183549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:28.183868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.183868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.184067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:28.184185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.184185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.185858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.186325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.187107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.187547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.210270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8f493ad0-b763-4d0d-9bbd-884ab8191a94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042681 Jul 20 15:21:28.210785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1263/1945] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:28.215117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:28.215710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:28.221535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.221954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.222704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:28.223162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:28.249633 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.258078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e470b764-4d33-43f6-a191-27385a7f6a2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083442 Jul 20 15:21:28.259149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 683/1946] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:28.284305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:28.292355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-945e83c9-2003-482f-9146-a2e335105f5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080608 Jul 20 15:21:28.292919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1264/1947] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:29.183241 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:29.183880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:29.191399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.191599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:29.192041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.192179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:29.192628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.193061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.198691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.199182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.199990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.200421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.392850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.397723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.401370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d56fbc77-d266-4d63-8d80-83bf53cc8eb5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.222276 Jul 20 15:21:29.402011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 684/1948] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:29 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:29.407314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.407535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:29.408429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:29.408813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.410121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d8a5857b-5def-4e8a-aba1-64becff6f309 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.222774 Jul 20 15:21:29.410829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1265/1949] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:29.415279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:29.415899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:29.423278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.423752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.424517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.424944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.427662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.428113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.428878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.429309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.459163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f0829c21-1645-4c68-8b20-a54e80747e94 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056002 Jul 20 15:21:29.459695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 685/1950] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:29.464115 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:29.464716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:29.470829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.471318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.472090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:29.472715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:29.489929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.498776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1bc1d277-7db2-4025-a19f-66adf6fe142a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087060 Jul 20 15:21:29.499410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1266/1951] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:29.544856 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:29.553268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9c51dd33-18cf-421c-a31f-3014574e799c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092629 Jul 20 15:21:29.553810 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 686/1952] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:30.422284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:30.423343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:30.430479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:30.431084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:30.431577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.432026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.432766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.433206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.437945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.438395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.439195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.439630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.622704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.627015 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.631486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-392c78c9-1150-44c1-ac49-6e8583cc6dd6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.215584 Jul 20 15:21:30.632070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1267/1953] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:30 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:30.635970 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.637171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:30.637478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.638193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:30.638711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a0e72876-39ed-44bc-9333-05d93b93b2ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.213012 Jul 20 15:21:30.639506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 687/1954] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:30.645389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:30.645389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:30.647591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.648236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.649174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.649763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.651296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.651771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.652560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.652959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.680851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7e3940f5-0a7f-4a40-a756-0092c02e7a3e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047639 Jul 20 15:21:30.681335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1268/1955] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:30.686090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:30.686739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:30.692776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.693251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.694018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:30.694441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:30.718830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.727392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-34aa3b9a-2604-4281-91f3-49289ebd0aa8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087033 Jul 20 15:21:30.727968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 688/1956] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:30 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:30.757263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:30.765513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-94587661-9c1c-47d4-92c7-755ffe611cd4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083220 Jul 20 15:21:30.766073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1269/1957] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:31.655128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:31.656419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:31.661005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:31.662056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:31.668206 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.668990 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.670240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.670967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.672728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.673475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.674685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.675505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.881999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:31.885444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:31.890729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2cd0016b-12f1-41db-83c1-ac09f56d6351 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.242306 Jul 20 15:21:31.891780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 689/1958] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:31 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:31.894695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:31.896311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:31.896790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:31.897637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85372851-55ab-4a95-86fc-a1f02b82aabb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.243049 Jul 20 15:21:31.897750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:31.901897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1270/1959] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:31.903025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:31.903534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:31.904062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.904574 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.905420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.905798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.912517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.913023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.913753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.914264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.937907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7f806bc9-6155-4c6f-954a-6a18af7edc0c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044880 Jul 20 15:21:31.938492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 690/1960] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:31 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:31.943216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:31.943824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:31.950204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.950659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.951451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:31.951886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:31.979930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:31.988048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f75efa19-b5ca-451c-9bcd-9a0b5e2b0578 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088745 Jul 20 15:21:31.988593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1271/1961] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:32.014766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:32.023499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ce6a8cab-4e54-4cec-b48f-a66be1318223 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084045 Jul 20 15:21:32.024093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 691/1962] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:32.911949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:32.913004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:32.918513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:32.919110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:32.920615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:32.921068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:32.921845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:32.922270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:32.925824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:32.926395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:32.927227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:32.927659 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.033933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:21:33.034118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 (ip 10.222.0.15) !!! Jul 20 15:21:33.124970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.127740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.131207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:33.137909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-947eaddd-1095-41ed-82cd-2be6160f794c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.232437 Jul 20 15:21:33.138111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.138564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1272/1963] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:32 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:33.139560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.140726 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9e87d9f5-f5fc-4e98-8f7f-f26dc52b209a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.226970 Jul 20 15:21:33.141450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 692/1964] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:33.143439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:33.144418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:33.145771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:33.146383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:33.150640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.151094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.151975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.152402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.153166 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.154512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.160725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.161167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.184593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-10ff0acb-9b31-4279-8453-e95a47236a0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044805 Jul 20 15:21:33.186308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1273/1965] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:33.189526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:33.190124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:33.196036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.196475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.197237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:33.197648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:33.224033 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.232125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-60685e20-3e47-4151-acf9-2dc29dddbb19 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089883 Jul 20 15:21:33.232698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 693/1966] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:33 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:33.262275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:33.270486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-663a2d7b-9492-4908-998c-569d6d6aec82 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084442 Jul 20 15:21:33.271039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1274/1967] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:34.165767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:34.166390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:34.166786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:34.166876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:34.173267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.173439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.173720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.173884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.174488 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.174640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.174924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.175088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.368532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.372189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.377483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.378972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.380196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-dfd8a069-3805-46c8-b154-91660345e454 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.218336 Jul 20 15:21:34.380424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-15b85bbf-823a-4012-a632-91b97c0469ae tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.218388 Jul 20 15:21:34.380754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 694/1968] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:34.380964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1275/1969] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:34 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:34.385522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:34.386090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:34.392007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.392465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.393247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.393717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.394977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:34.395975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:34.402077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.402527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.403314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.403745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.434896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cd2be3c6-d111-4938-8310-3db48679d4ce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043923 Jul 20 15:21:34.435523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 695/1970] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:34 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:34.440173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:34.440778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:34.446852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.447365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.448136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:34.448569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:34.455523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.463680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4bfe54e0-0645-4510-967e-2fa6d82c9c65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081778 Jul 20 15:21:34.464270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1276/1971] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:34.518199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:34.530642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e110156a-91ba-4e15-9042-5251a00b4756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093817 Jul 20 15:21:34.531476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 696/1972] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:35.401011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:35.401696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:35.402071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:35.402194 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:35.408625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.408801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.409069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.409409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.409837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.410268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.410372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.410823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.603561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.606363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.612153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.613650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.614517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a64fd23f-b07c-4354-a21d-11e694d494c3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.219439 Jul 20 15:21:35.614828 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-14938235-ea44-4f1a-852b-e4d24f4d860d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.218919 Jul 20 15:21:35.615113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1277/1973] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:35 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:35.624546 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 697/1974] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:35.624751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:35.624751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:35.624929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:35.624929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:35.626971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.627449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.628205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.628634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.629918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.630369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.631255 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.631715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.645191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:35.660459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-60cef42a-2d6b-475e-bd74-f7853155911a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044241 Jul 20 15:21:35.661093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1278/1975] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:35 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:35.665627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:35.666267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:35.672175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.672614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.673373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:35.673807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:35.697589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.705555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e48c2ec8-7725-48eb-93ac-e3d5b332b22a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087163 Jul 20 15:21:35.706130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 698/1976] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:35.736518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:35.744891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8dc868c7-2bb6-4766-9c3c-73f0adb95e5b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082836 Jul 20 15:21:35.745445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1279/1977] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:36.636063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:36.636744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:36.637481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:36.638257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:36.643629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.644079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.644838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.645289 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.646131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.646577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.647339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.647793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.841141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.844483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.851820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-73bbd88c-9f47-4984-9d98-7be0511533ea tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.221477 Jul 20 15:21:36.852558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 699/1978] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:36 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:36.855990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.857250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:36.857517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.858222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:36.858688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a5302cbb-4fa8-474b-90b5-6a9d73116134 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.226254 Jul 20 15:21:36.861278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1280/1979] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:36.864436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.864908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.865046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:36.865654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:36.865785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.866049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.872387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.872835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.873598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.874030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.897624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-951102f5-3b28-4d0d-a49b-a768918c28ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044015 Jul 20 15:21:36.898231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 700/1980] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:36.903291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:36.903900 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:36.910171 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.910616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.911414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:36.911849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:36.938017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.946246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9433dc38-d98c-4199-99bd-23b7635b2f4d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085880 Jul 20 15:21:36.946822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1281/1981] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:36 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:36.972208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:36.980211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f0e2214b-ae02-405d-ba7b-a9195fede7d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080498 Jul 20 15:21:36.980977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 701/1982] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:37.873753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:37.874769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:37.881566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:37.882577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:37.885774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:37.886520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:37.887878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:37.888580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:37.893020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:37.893800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:37.895010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:37.895764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.083666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.088304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.096380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-899f2c71-7d75-4415-9654-8355474bcadb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.229237 Jul 20 15:21:38.097131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1282/1983] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:37 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:38.107268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.107426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:38.107520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.107630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:38.107716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6948ae4f-b882-445e-9f98-71263431c71b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.230792 Jul 20 15:21:38.107716 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 702/1984] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:38.110616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:38.111244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:38.113016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.113518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.114379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.114881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.117535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.117990 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.118759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.119229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.145231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b55873d-c1fd-475b-b8ec-bd7cfb862bb0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046944 Jul 20 15:21:38.145758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1283/1985] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:38.150266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:38.150874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:38.157015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.157437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.158197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:38.158630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:38.180584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.188496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-10bd39fb-26a7-4d90-ab4c-e752cd2aa2c9 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081255 Jul 20 15:21:38.189072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 703/1986] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:38 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:38.219610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:38.228017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-666ff6ab-119a-4592-8bda-5c68f46a09c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081321 Jul 20 15:21:38.228582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1284/1987] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:39.120104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:39.120667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:39.124599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:39.125197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:39.127674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.128127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.128897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.129319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.131812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.132263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.133010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.133442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.331607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.334730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.340375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.341940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.342860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a3298e39-2514-45f0-a105-104dd3ee4f58 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.228784 Jul 20 15:21:39.343278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-452f86d0-077b-4afb-9189-18036edaef1a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.222549 Jul 20 15:21:39.343823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 704/1988] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:39 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:39.344137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1285/1989] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:39.348498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:39.349111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:39.355963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.356483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.357306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.357835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.358433 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:39.359451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:39.365755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.366202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.367310 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.367733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.397511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1024591c-cdba-427b-8d64-30bfa26a6407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042823 Jul 20 15:21:39.398077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 705/1990] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:39.402630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:39.403276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:39.409370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.409821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.410584 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:39.410996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:39.423626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.432091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f693551a-f031-4e8b-8247-924504db17ee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087015 Jul 20 15:21:39.432681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1286/1991] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:39.475585 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:39.484529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6206861d-2cae-4495-bc08-2ec5e5e5b3ef tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085443 Jul 20 15:21:39.485116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 706/1992] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:40.364760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:40.365761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:40.366400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:40.366563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:40.377219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.377522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.377950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.378217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.379238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.379563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.380007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.380201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.577901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.580376 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.586584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2e4b1449-dc98-4674-a562-c9a7f37da02a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.228310 Jul 20 15:21:40.587273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1287/1993] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:40 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:40.589933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.591509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.591982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:40.592681 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ffb5aaea-c3b5-4339-803c-55cfe12ddff9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.234011 Jul 20 15:21:40.592962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:40.594898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 707/1994] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:40.597823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:40.598418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:40.600102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.600640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.601463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.601976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.604823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.605257 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.606025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.606449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.632261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e06a76b3-8ca2-4b99-8b54-726c38bd623d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043818 Jul 20 15:21:40.636095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1288/1995] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:40 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:40.637051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:40.637654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:40.643829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.644277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.645038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:40.645469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:40.668366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.676611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4d11d12e-53ca-4391-9ebc-05164527eaef tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082110 Jul 20 15:21:40.677244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 708/1996] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:40 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:40.719965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:40.728215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a08a5ba1-7209-4f9a-8de3-265c231aab2d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094571 Jul 20 15:21:40.728778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1289/1997] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:40 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:41.609239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:41.610077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:41.615197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:41.616971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:41.624564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.625035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.625884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.626441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.629729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.631124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.631124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.631409 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.654144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:41.825719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.828863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.835109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.836605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.837390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-09d9e858-51cd-4a60-81c2-aacab56ef98d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.235703 Jul 20 15:21:41.837802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-728a559c-23c4-49fa-8f89-b89d8fc30a79 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.229460 Jul 20 15:21:41.838318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 709/1998] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:41 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:41.843217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:41.844219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:41.844495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1290/1999] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:41.849584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:41.850344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:41.852326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.852836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.853604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.854038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.856515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.856961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.857719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.858150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.883959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c19ba9f5-69ff-4e98-87e2-a556fd075c6d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044354 Jul 20 15:21:41.884446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 710/2000] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:41.889149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:41.889761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:41.895973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.896420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.897185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:41.897641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:41.921533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.930732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-936b72d3-cd81-4690-a3d1-e4bfee21bd73 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.084918 Jul 20 15:21:41.931662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1291/2001] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:41 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:41.964383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:41.973365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ef9907a8-2e99-4c95-9a5f-7112599d505e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087925 Jul 20 15:21:41.973956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 711/2002] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:42.857100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:42.857898 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:42.858411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:42.858529 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:42.866459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:42.867313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:42.867514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:42.867914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:42.868530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:42.868858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:42.869025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:42.869296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.065016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.068159 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.073285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92af98cf-a599-4771-b6e5-224a8e27dd3c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.222393 Jul 20 15:21:43.073910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1292/2003] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:43.076850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.078359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.078733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:43.079562 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b7b698d8-a590-4bd5-b18a-fcc088de891e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.227015 Jul 20 15:21:43.079910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:43.083466 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 712/2004] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:43.084995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:43.085608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:43.086364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.087175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.087856 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.088528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.091971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.092424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.093258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.093696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.117986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57b15e26-999a-427c-8d7b-c1f56cfab74f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042825 Jul 20 15:21:43.118537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1293/2005] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:43 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:43.122879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:43.123511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:43.129495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.129951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.130709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:43.131173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:43.158732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.167145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-594b260a-3eae-4568-998e-dab9a7d66453 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085579 Jul 20 15:21:43.167694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 713/2006] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:43.191372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:43.199566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-91899898-f84c-4de6-9306-c0fe3d751e1c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080020 Jul 20 15:21:43.200137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1294/2007] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:44.094876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:44.095948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:44.101837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:44.102750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:44.105366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.105811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.106599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.107020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.109545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.109995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.110799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.111220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.115690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:44.299084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.302441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.307387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3c2e3fa3-1811-4e54-bc3a-c4630c01a5dd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.219179 Jul 20 15:21:44.307965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 714/2008] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:44 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:44.311424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.312494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:44.312901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.313468 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:44.314054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9626ef3d-d3fb-4425-b841-eab53b1516c6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.218916 Jul 20 15:21:44.317383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1295/2009] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:44.319292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:44.319435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.319821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.319938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:44.320575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.321000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.321666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:44.327699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.328125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.328901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.329325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.350287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e80826ab-c772-471a-8182-9b0e4dbbe3a4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041205 Jul 20 15:21:44.350975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 715/2010] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:44.355138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:44.355744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:44.361661 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.362111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.362866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:44.363327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:44.391015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.399278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44a34aa3-b968-4ae3-a349-aa2f9bdc5d69 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083320 Jul 20 15:21:44.399843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1296/2011] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:44 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:44.423439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:44.431457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8438147d-bc75-459e-96b8-7057c81c858c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.079628 Jul 20 15:21:44.431988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 716/2012] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:45.324522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:45.325133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:45.332212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.332670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.333041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:45.333531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.333638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:45.333848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.335991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:45.341632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.342073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.342867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.343328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.784229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.788791 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.796683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7ecea10f-a6db-48d7-b887-f0476a6ba5b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.476027 Jul 20 15:21:45.797294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1297/2013] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:45 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:45.797753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.799289 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.800454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2754b190-f508-4c42-a7f1-49e4b5e9b022 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.471378 Jul 20 15:21:45.801161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 717/2014] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:45.802078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:45.803071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:45.805281 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:45.805915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:45.809390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.809841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.810600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.811030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.811858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.812308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.813070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.813502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.841360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7bd153c-e4b0-4ba9-b80a-3dddc6b45ff4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042998 Jul 20 15:21:45.842372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1298/2015] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:45.848682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:45.849285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:45.855671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.856156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.856925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:45.857360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:45.875517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.883556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ac20f8e8-9d4f-4144-bdc2-ac2bd9c5a5e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081526 Jul 20 15:21:45.884130 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 718/2016] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:45.921436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:45.929522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-82746e4c-5e61-4e38-81a7-7893d670cfa3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086351 Jul 20 15:21:45.930060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1299/2017] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:46.827496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:46.829260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:46.830687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:46.831863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:46.839227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:46.839680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:46.840380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:46.840512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:46.840837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:46.840960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:46.841599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:46.842033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.031492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.034584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.040662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ea230c02-e881-430d-8a47-30974b11281b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.219995 Jul 20 15:21:47.041241 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 719/2018] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:46 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:47.044379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.045753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:47.045897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.046758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:47.047019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84c79ce7-f145-4528-89b9-38f796152179 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.223824 Jul 20 15:21:47.047785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1300/2019] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:47.052686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.053135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.053893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.054323 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.062047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:47.062654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:47.068894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.069347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.070107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.070509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.083328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c6913bdb-1055-41d8-8bc7-55b87258c216 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040953 Jul 20 15:21:47.083804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 720/2020] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:47.088168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:47.088767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:47.094505 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.094953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.095747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:47.096185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:47.136274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.144479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0ca7bc9-125b-4439-8466-36347830aa20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085994 Jul 20 15:21:47.145037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1301/2021] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:47 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:47.162537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:47.170945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-82f135e7-a7c0-4087-8610-1b6b7039abbf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086220 Jul 20 15:21:47.171665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 721/2022] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:48.061749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:48.062694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:48.068384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:48.069393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:48.072791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.073233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.073996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.074424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.076387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.076845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.077590 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.078032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.269144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.273625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.278844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34edf4f6-7be1-412b-9324-5b95ea08f805 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.224000 Jul 20 15:21:48.279420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1302/2023] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:48 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:48.282648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.283871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:48.284171 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.284829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:48.285393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5b66fed2-4965-447d-8208-29aeef7c95b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.223422 Jul 20 15:21:48.286116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 722/2024] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:48.290617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:48.291254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:48.296515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.296969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.297141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.297491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.297735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.298168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.298333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.298662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.330373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-13d9abb3-7ae0-41ff-b6e3-1dd984031211 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049763 Jul 20 15:21:48.330824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1303/2025] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:48.335368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:48.335970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:48.342527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.342992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.343783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:48.344218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:48.360845 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.369460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-167f21d2-0b37-406f-8937-e3e1a8871810 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082516 Jul 20 15:21:48.370029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 723/2026] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:48 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:48.405480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:48.414001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53f074ee-a709-4bd5-acdf-d7012e4aa0b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082173 Jul 20 15:21:48.414679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1304/2027] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:49.300256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:49.301222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:49.308032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:49.309088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:49.312449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.313229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.314493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.315242 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.319849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.320580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.321658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.322087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.513787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.517786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.522138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-80b0a2f7-706f-46cd-b634-47c75c278dc7 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.228342 Jul 20 15:21:49.522713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 724/2028] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:49.526884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.527385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:49.528385 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:49.528514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.529547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-37322e81-ec1e-4d05-a479-9f7b8312eac8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.228307 Jul 20 15:21:49.539240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1305/2029] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:49.539451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.539638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:49.539744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.539841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:49.539936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.539936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.542445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.542930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.543753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.544191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.565641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a6a0a84a-54a0-47bb-9843-9f3c4d3a86bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041783 Jul 20 15:21:49.566098 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 725/2030] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:49.570410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:49.571104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:49.576912 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.577353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.578115 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:49.578548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:49.582495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:49.606573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.614548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e54a2d34-abfb-43be-a04e-4c7ea2816ebf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083148 Jul 20 15:21:49.615131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1306/2031] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:49 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:49.643072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:49.651688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fa3178d8-6e34-4dea-9345-6f29ee166fd8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084630 Jul 20 15:21:49.652244 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 726/2032] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:50.542656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:50.543851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:50.551715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:50.552778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:50.554912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.555741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.556507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.556943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.560663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.561118 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.561881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.562319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.751763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.755625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.760683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6995d325-4990-43c4-a28d-6c3c51718984 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.224518 Jul 20 15:21:50.761264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1307/2033] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:50 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:50.765891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:50.766821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:50.773223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.773668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.774455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.774613 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.774867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.776054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.777238 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-aca22179-6d4d-43d7-93fc-f4786f0244d7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.232070 Jul 20 15:21:50.777797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 727/2034] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:50.782291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:50.782894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:50.788882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.789335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.790170 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.790610 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.805112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bec620ee-c950-433b-9dfc-7c24f3a87aa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042773 Jul 20 15:21:50.805747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1308/2035] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:50.812452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:50.813056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:50.819252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.819701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.820451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:50.820925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:50.854846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.863186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3af47f97-77f9-414d-b2b9-0df0b1f37c5d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.084370 Jul 20 15:21:50.863853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 728/2036] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:50 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:50.882680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:50.890859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9d797276-0c86-456d-87ca-a3e6d12fd46c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084170 Jul 20 15:21:50.891463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1309/2037] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:51.781600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:51.782457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:51.792153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:51.792599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:51.793367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:51.793812 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:51.796602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:51.797377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:51.805657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:51.806129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:51.806900 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:51.807374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.010295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.017803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.018542 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1461b57c-5431-45a2-a72f-cf6e874b1a9d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.243436 Jul 20 15:21:52.019105 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 729/2038] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:51 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:52.023872 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:52.024957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:52.032980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.033364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.034129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.034564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.035394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.036880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.038042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02114fb2-8521-4f2d-a019-617b7a45670f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.245909 Jul 20 15:21:52.048703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1310/2039] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:52.048703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:52.048703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:52.050205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.050705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.051535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.051965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.065078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f74d71b7-716c-4fe6-bc85-66653486a06a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044901 Jul 20 15:21:52.069135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 730/2040] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:52.070096 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:52.070678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:52.076787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.077231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.077994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:52.078423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:52.124901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.128511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:52.134372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2af53f94-09a7-4891-8eed-0e0f044c2a3d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094598 Jul 20 15:21:52.134955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1311/2041] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:52 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:52.144875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:52.153360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2b889146-7d71-4c05-97a3-c624b664bbfd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086779 Jul 20 15:21:52.153930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 731/2042] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:53.147990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:53.148920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:53.155690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.156132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.157039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.157494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.186849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:53.187502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:53.193750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.194202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.194972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.195438 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.304505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.313213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-258e57de-5414-4329-9f46-1e1e16af7595 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.280169 Jul 20 15:21:53.313790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1312/2043] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:53.390257 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.399202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.400695 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.401885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4aa0eedd-cdbb-4777-975f-c859f74e890c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.349124 Jul 20 15:21:53.402446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 732/2044] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:53.406792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:53.407424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:53.413309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.413756 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.414546 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.414982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.458088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:53.459105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:53.465480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.465928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.466681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.467135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.484222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.492279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1e250599-50be-46a6-9b07-75332b18e1f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088862 Jul 20 15:21:53.492963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 733/2045] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:53 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:53.503372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cc923eea-d210-4c48-ba69-e5713f3c3c39 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.188513 Jul 20 15:21:53.503946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1313/2046] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:53.632142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:53.632747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:53.638775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.639250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.639998 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:53.640426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:53.705645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:53.718427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8c8c6aae-cbdf-4c59-9438-682bb51bf304 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.224315 Jul 20 15:21:53.719012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 734/2047] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:53 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:54.334358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:54.335433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:54.343194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.343664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.344456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.344909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.420443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:54.421047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:54.427988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.428440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.429210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.429646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.514165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.522607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6425340e-d0b3-4dff-9342-944d89c8a9cd tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.194796 Jul 20 15:21:54.529149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1314/2048] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:54 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:54.529149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:54.529149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:54.535587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.536134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.536905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.537357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.590091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.599088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.600611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.601758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-647b562f-13f6-4737-8a06-e956e879f1ca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.185282 Jul 20 15:21:54.602399 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 735/2049] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:54.606704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.606941 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:54.607985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:54.614041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.614493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.614995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0deaebbb-49b6-4faf-80bd-256bd459190c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.090837 Jul 20 15:21:54.615288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.615608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1315/2050] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:54 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:54.615836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.647023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ea128c31-f243-46e5-a05e-170eade3478c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043615 Jul 20 15:21:54.647577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 736/2051] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:54.742219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:54.743263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:54.753317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.754110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.755362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:54.756102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:54.819325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:54.828148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0a28e551-fe15-4e6e-b451-1337e744c1ff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092816 Jul 20 15:21:54.828686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1316/2052] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:54 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:55.539749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:55.540331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:55.550759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.551514 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.552750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.553451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.622415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:55.622988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:55.629938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.630397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.631197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.631637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.742398 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.750877 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b45c935c-0ff1-488a-93d3-3806a0637a00 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.215150 Jul 20 15:21:55.751512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 737/2053] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:55 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:55.756060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:55.756665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:55.762775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.763263 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.764028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.764460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.787079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.795730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.797244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.798448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f65286d3-2be3-4cf5-bea6-74f1615821e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.180082 Jul 20 15:21:55.799017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1317/2054] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:55.803641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:55.804533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:55.811039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.811531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.812478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.812915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.826720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.834721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-50649630-c9e5-4fa1-8f96-fc69c2ec34e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082240 Jul 20 15:21:55.835372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 738/2055] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:55.844266 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:55.844773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0f887bf5-d6ca-4eae-aeda-bb349609a565 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044726 Jul 20 15:21:55.844891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:55.845396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1318/2056] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:55.850832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.851329 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.852101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:55.852537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:55.911956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:55.922867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bf409ebc-8e08-4c21-b489-6f01e694b13b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081894 Jul 20 15:21:55.923802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 739/2057] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:56.772416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:56.773312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:56.784196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.784942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.786196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.786915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.817302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:56.818523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:56.825550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.826004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.826891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.827360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.972167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:56.980757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-56ddb5dd-af97-457f-b1ed-3e078def6840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.214951 Jul 20 15:21:56.981362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1319/2058] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:56.985835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:56.986476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:56.992990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.993443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.994209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:56.994647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:56.998336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:57.006981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:57.008577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:57.009729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1782031f-cad7-46f4-9559-41704712b173 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.196473 Jul 20 15:21:57.010299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 740/2059] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:57.018790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:57.019831 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:57.026152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:57.026664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:57.027544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:57.027976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:57.057565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0446744b-970d-46c7-91ec-340426e48e9e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046237 Jul 20 15:21:57.058147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 741/2060] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:57.059117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:57.062491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:57.063120 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:57.067386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b80e6b85-08f6-49c7-b63e-3f8fcef5d9f5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085025 Jul 20 15:21:57.068032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1320/2061] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:57.068938 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:57.069389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:57.070150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:57.070579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:57.130972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:57.134374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:21:57.139947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-07a9e9f9-3f29-4073-a2e9-88b9729f416c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080867 Jul 20 15:21:57.140518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 742/2062] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:58.002253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:58.003252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:58.014503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.015284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.016526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.017232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.029368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:58.030108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:58.037064 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.037510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.038283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.038825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.242237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.248553 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.250762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fd2af6f5-4a1d-4d74-8a1a-e5ce2decabc4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.254891 Jul 20 15:21:58.251376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1321/2063] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:57 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:58.255946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:58.256826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:58.257361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.259730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.260548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-980f7078-0d5b-4c25-b2cb-fd5806827fd0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.235440 Jul 20 15:21:58.261262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 743/2064] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:58.262820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.263315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.264068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.264478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.265723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:58.266334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:58.272304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.272749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.273567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.273944 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.294731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d37b8c48-a8be-4405-8211-bca33d8411d7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042309 Jul 20 15:21:58.295302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1322/2065] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:58.299670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:58.300274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:58.305994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.306439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.307226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:58.307663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:58.340065 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.348300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-77d73b5b-d988-4bcd-9fdb-8cfeb1a28dca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086208 Jul 20 15:21:58.348882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 744/2066] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:58 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:58.373328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:58.381477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a188f7bf-76b6-4d1b-b10f-fdac6c97774b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085239 Jul 20 15:21:58.382042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1323/2067] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:59.272363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:59.273374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:59.283282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:59.284371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:59.284662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.285332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.286626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.287393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.293108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.293559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.294318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.294750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.485271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.488479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.496534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.497643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3e57ada6-7839-4c2a-98f0-df7a2fcb1873 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.231635 Jul 20 15:21:59.498154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.498558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 745/2068] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:59 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:59.499405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b3d72f0b-b48a-4d0b-ad99-133fc5cb98c8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.222747 Jul 20 15:21:59.500130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1324/2069] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:21:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:21:59.503084 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:59.504039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:59.510003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.510455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.511234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.511662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.511903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:59.512506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:59.518881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.519359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.520105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.520530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.540359 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c2d6506b-f9cb-4ea9-a667-f6581e9730d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040837 Jul 20 15:21:59.541089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 746/2070] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:21:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:21:59.545204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:21:59.545804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:21:59.551586 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.552028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.552792 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:21:59.553222 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:21:59.582661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.590918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-28f23cf0-fcff-4488-bb1a-42d35a9e78f7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082717 Jul 20 15:21:59.591538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1325/2071] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:21:59.617175 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:21:59.625297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f7c45cf9-3d63-41c3-8278-82dfbf9f638a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083510 Jul 20 15:21:59.625852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 747/2072] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:21:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:00.515985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:00.516952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:00.520558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:00.521612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:00.526990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.527837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.529085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.529805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.532618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.533325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.534616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.535378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.734788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.742784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.744019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ae3d731e-a2a9-4ae5-a560-fdd42d9ed225 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.233356 Jul 20 15:22:00.744972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1326/2073] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:00 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:00.749358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:00.750321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:00.751996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.753485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.755158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-625e8e54-8726-4131-a931-31006be85131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.240748 Jul 20 15:22:00.755832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 748/2074] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:00.756801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.757249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.757994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.758560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.760575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:00.761182 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:00.767426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.767874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.768634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.769067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.792468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7fe4e7f-a016-4986-9501-e35a5660caed tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046580 Jul 20 15:22:00.793046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1327/2075] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:00.797575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:00.798268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:00.804759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.805301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.806054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:00.806484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:00.838565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:00.852547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.863018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c301211e-8dab-4564-a900-4b4083732ac3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.105913 Jul 20 15:22:00.864113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 749/2076] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:00.892572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:00.901302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ce80acb-f308-4a0a-8699-f2f77e8f3ce9 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.107280 Jul 20 15:22:00.901871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1328/2077] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:01.765775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:01.766612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:01.776325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:01.776785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.776785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.776986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:01.777555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.777991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.783951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.784397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.785159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.785587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.973161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:01.976072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:01.981393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-49f50804-ae6f-433b-aa9c-c6d8928bd3a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.222296 Jul 20 15:22:01.981972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 750/2078] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:01 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:01.985172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:01.986666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:01.986809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:01.987679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:01.987883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8bc2c5bd-c4bb-44db-8e82-3b6903e5b521 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.216907 Jul 20 15:22:01.988576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1329/2079] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:01.993054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:01.993674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:01.994299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.994766 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.995680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:01.996131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:01.999790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:02.000242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:02.000991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:02.001420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:02.027195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c85da2e8-9c1e-458d-a971-7496600c0b68 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044044 Jul 20 15:22:02.027688 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 751/2080] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:02.034496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:02.035187 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:02.041414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:02.041851 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:02.042608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:02.043032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:02.061958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:02.069976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-344f8921-7267-4ded-895d-4505bfde0543 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.080629 Jul 20 15:22:02.070610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1330/2081] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:01 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:02.105455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:02.117631 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-278de567-96dc-4685-850d-9f2bc3df7f90 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088750 Jul 20 15:22:02.118456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 752/2082] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:03.002779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:03.003817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:03.011677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:03.012365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:03.012696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.013139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.013910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.014337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.019097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.019579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.020336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.020767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.232554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.235566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.240967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c008a10-5580-4087-93e7-fdc7dfad3f21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.244584 Jul 20 15:22:03.241893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1331/2083] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:02 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:03.244939 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.246435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.246575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:03.247551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:03.247667 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-603d203a-d036-4eda-a473-ae44e206b106 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.240568 Jul 20 15:22:03.248364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 753/2084] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:03.252744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:03.253351 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:03.258499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.258950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.259392 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.259755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.259899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.260177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.260611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.261049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.289602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-088feba6-3233-4828-8257-7cc9d7b61f41 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046625 Jul 20 15:22:03.290171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1332/2085] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:03.294545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:03.295172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:03.301094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.301545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.302307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:03.302738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:03.325396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.333943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-722734ef-7038-41bb-97bd-e0947fac504f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.084703 Jul 20 15:22:03.334516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 754/2086] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:03.363622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:03.372161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80124e81-e9e9-4713-8e31-1f780af78614 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080990 Jul 20 15:22:03.373044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1333/2087] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:04.261651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:04.262449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:04.267778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:04.268562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:04.271239 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.271800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.272794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.273340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.276834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.277403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.278357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.278902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.471453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.475213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.480822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.482336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.483516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4a5635ff-32f7-4340-a2cf-a52f3cc5c8ec tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.220855 Jul 20 15:22:04.484168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5bb570bb-387f-4e44-9082-d54cb81b602d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.227876 Jul 20 15:22:04.487118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1334/2088] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:04.487823 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 755/2089] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:04 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:04.498014 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:04.498465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:04.498616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:04.505093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.505541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.506311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.506739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.508109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:04.514671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.515073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.515834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.516278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.546927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-266cfe5a-9a42-4c4b-9aa1-319cb877150f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052202 Jul 20 15:22:04.547526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1335/2090] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:04.551824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:04.552419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:04.558821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.559288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.560044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:04.560475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:04.582867 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.591316 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6c7a9c95-7a4f-4800-8bd6-263401c3cf94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.104936 Jul 20 15:22:04.592218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 756/2091] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:04.627509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:04.635727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c0090883-3533-4c66-800d-6274524661b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087304 Jul 20 15:22:04.636291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1336/2092] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:05.501953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:05.502464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:05.502836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:05.502941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:05.509623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.509820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.510070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.510249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.510833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.510982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.511302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.511475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.701783 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.710109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.715517 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-13c2c958-e9c8-4d55-9108-7417e1b32c32 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.214821 Jul 20 15:22:05.716427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 757/2093] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:05 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:05.723121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.723381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:05.724628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:05.725082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.726942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b33732c1-1873-42e9-8da1-13b8a7a65b24 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.228117 Jul 20 15:22:05.729532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1337/2094] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:05.731521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.731885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:05.732026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.732488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:05.732722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.733147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.738421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.738869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.739634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.740058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.763425 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c58a94f9-fd75-4e25-97c0-56389f1ea8ba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045800 Jul 20 15:22:05.764020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 758/2095] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:05.768358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:05.768972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:05.775151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.775580 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.776342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:05.776776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:05.795318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:05.802206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.810962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9ca81f34-4452-4fd1-922e-62c2d34c4cc4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082429 Jul 20 15:22:05.811753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1338/2096] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:05.841555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:05.849744 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f2613ef7-4ee3-4bc2-97ab-00d10e21abec tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084826 Jul 20 15:22:05.850348 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 759/2097] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:06.741629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:06.742554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:06.748704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:06.749975 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:06.754688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.755511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.756888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.757790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.759723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.760243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.761000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.761427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.955343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:06.957813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:06.963965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-edee455b-2da1-4346-b43d-beb7ce7b9e21 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.229985 Jul 20 15:22:06.964555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1339/2098] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:06 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:06.970852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:06.970852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:06.971106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:06.971223 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a7b79b0c-9d7a-4ea5-9ffa-cf2ea416495d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.227759 Jul 20 15:22:06.971323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:06.971405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 760/2099] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:06.975039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:06.975894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:06.976680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.977132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.977881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.978324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.982108 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.982556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:06.983357 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:06.983923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:07.008780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff7fdab3-d98f-4924-8fdc-8d1c94cd364c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043094 Jul 20 15:22:07.009261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1340/2100] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:07.013555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:07.014221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:07.020431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:07.020874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:07.021627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:07.022061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:07.045895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:07.054064 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6156246b-c487-41c2-8924-c097635291fd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082619 Jul 20 15:22:07.054632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 761/2101] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:07.083909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:07.092292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44580eeb-0bf9-41b9-ae5d-6d49c35f79d1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082091 Jul 20 15:22:07.092859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1341/2102] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:07 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:07.986573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:07.988383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:07.992722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:07.993729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:07.999861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.000602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.001860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.002635 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.004890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.005651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.006898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.007671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.196924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.199553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.205386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-08dcaf98-f066-4746-8fdf-4ee59b093b2b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.225176 Jul 20 15:22:08.205942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 762/2103] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:07 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:08.209221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.210467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:08.210851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.211582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:08.212066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0e252b8-0d2e-4b5f-a47d-08680274d131 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.226245 Jul 20 15:22:08.215098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1342/2104] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:08.217714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:08.217896 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.218131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.218315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:08.218925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.219410 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.224508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.224969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.225753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.226298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.248822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-75afb2e8-9306-44ab-8f2a-381b3135d9b8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041766 Jul 20 15:22:08.249321 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 763/2105] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:08.253720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:08.254335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:08.260552 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.261004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.261781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:08.262204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:08.288537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.296781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-26000cb8-960c-419f-80a6-120cc81da10e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082546 Jul 20 15:22:08.297351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1343/2106] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:08 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:08.326344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:08.334336 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0f9d7296-9fda-4bd7-8bbe-7c59393838cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084090 Jul 20 15:22:08.334924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 764/2107] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:09.222355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:09.222975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:09.230424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:09.231075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:09.234800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.235290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.236144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.236570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.237869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.238313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.239085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.239541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.475283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.483929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59888a04-39c2-4bce-af8e-7d26bb3d5a40 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.265410 Jul 20 15:22:09.486657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.487292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1344/2108] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:09 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:09.493106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:09.494151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:09.496405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.497913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.499088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d5ecb00c-c97f-40e2-8eff-a14b0bfb925e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.272669 Jul 20 15:22:09.499680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 765/2109] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:09.500480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.500936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.501699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.502126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.504005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:09.504612 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:09.510715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.511209 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.511981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.512446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.519304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:09.537462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-504a4aa2-f263-4a3c-aabe-c30cf09da40c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048729 Jul 20 15:22:09.538233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1345/2110] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:09.542950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:09.543599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:09.545394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:09.550783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.551264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.552036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:09.552466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:09.591330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.599656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-aeda2811-77ac-40d4-a20a-2eff85a73b6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.098757 Jul 20 15:22:09.600015 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 766/2111] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:09 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:09.625825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:09.634315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d9abd65-ff63-4231-be9f-4c7069095f5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.095181 Jul 20 15:22:09.634932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1346/2112] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:10.505589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:10.506605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:10.509537 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:10.515632 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.516078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.516659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:10.517092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.517260 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.517386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:10.524044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.524478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.525235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.525649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.711268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.714386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.719420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-38e4c081-ab64-4304-809d-0c78cd03ff41 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.220704 Jul 20 15:22:10.729060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 767/2113] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:10 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:10.729360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.729490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:10.729581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.729681 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:10.729767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59a9a3ee-49ac-46b3-bdb4-c462f08a91f7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.213534 Jul 20 15:22:10.729767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1347/2114] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:10.731867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:10.731987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.732311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.732524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:10.733068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.733498 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.738819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.739292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.740053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.740522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.763689 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3e6d3901-537f-4d9a-843b-5aac96c2adc5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042533 Jul 20 15:22:10.764172 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 768/2115] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:10.768526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:10.769131 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:10.775065 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.775526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.776293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:10.776735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:10.801353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.809569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81202e02-b840-46c5-afaa-df1bdecbe1a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081715 Jul 20 15:22:10.810198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1348/2116] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:10 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:10.842402 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:10.850502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b6d45d16-08c3-4e54-914f-9316e2ace75d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085394 Jul 20 15:22:10.851093 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 769/2117] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:11.741193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:11.742067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:11.749407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:11.750419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:11.753542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.754274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.755975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.756343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.760204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.762368 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.763173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.763643 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.898234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:11.906834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c51e857e-4f5c-4b84-b70d-6bd6f0109202 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.172580 Jul 20 15:22:11.907503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1349/2118] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:11 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:11.912162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:11.913143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:11.921765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.922462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.923701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.924323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.966031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5ca7677a-1993-44f4-8594-ef44a9d21ee8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057458 Jul 20 15:22:11.966623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1350/2119] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:11.971261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:11.971864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:11.978038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.978476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:11.979272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:11.979696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.060060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:12.068143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b1bc5e99-aadd-42a9-a48c-43be0401f853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.100363 Jul 20 15:22:12.068722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1351/2120] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:11 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:12.073292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:12.073903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:12.080086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.080538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.081321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.081519 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:12.081749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.090149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:12.091652 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:12.092794 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-012a809c-e616-4ea4-bd97-d1d3352b0e74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.350486 Jul 20 15:22:12.093525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 770/2121] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:12.143129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:12.151398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2edc95d5-987f-4e0d-95f3-25653d01f879 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081686 Jul 20 15:22:12.151962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1352/2122] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:12.927259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:12.928298 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:12.935935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.936375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.937127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.937548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.978080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:12.979019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:12.985659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.986110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:12.986870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:12.987329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.017786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8cec09b5-72b5-407e-a290-a91cc8677b24 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043755 Jul 20 15:22:13.018358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1353/2123] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:13.068008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.079747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f0fc0f4c-ae33-4a41-838f-5466d087566a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.158871 Jul 20 15:22:13.080333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 771/2124] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:12 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:13.087628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:13.088241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:13.094669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.095150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.095911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.096341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.110796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:13.111451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:13.118178 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.118621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.119606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.120038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.169321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.178046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6a9effe6-f4e7-40dc-9d14-5545432ba256 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.095384 Jul 20 15:22:13.178555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1354/2125] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:13.182996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:13.183648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:13.189528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.190008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.190763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:13.191224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:13.259394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.267574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5105dcc6-28ca-4703-a14b-542f29a16a95 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088076 Jul 20 15:22:13.268151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1355/2126] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:13.282472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.293536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.295002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:13.296193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-01e1f960-0bf7-4d00-a97a-702741e087b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.189229 Jul 20 15:22:13.296727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 772/2127] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:14.033691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:14.035243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:14.043803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.044250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.045003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.045434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.077263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-315ba515-3651-4722-aec4-a2493e4fbda0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050286 Jul 20 15:22:14.077805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1356/2128] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:22:14.097905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:14.098522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:14.105854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.106313 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.107073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.107522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.195471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:14.196087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:14.202898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.203408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.204210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.204643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.252115 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.260566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-25b452c4-a259-4c06-945f-21400a7e1b68 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166496 Jul 20 15:22:14.261129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 773/2129] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:14 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:14.267984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.279352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7611fa09-8538-46dc-b4ce-efdf7923cb75 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087845 Jul 20 15:22:14.279876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1357/2130] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:14.284520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:14.285188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:14.291844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.292295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.293057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.293495 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.313885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:14.314487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:14.320865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.321311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.322059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:14.322492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:14.354928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.363871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a93c7e88-9722-4194-8692-2d12c82ae58a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083107 Jul 20 15:22:14.364414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 774/2131] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:14.449374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.457845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.459366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:14.460530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d151e89d-716e-453d-92fb-1c4fe92548b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.150481 Jul 20 15:22:14.461209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1358/2132] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:22:15.093412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:15.094999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:15.106110 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.107150 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.108236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.108952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.142236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-63ba81e7-392c-42ab-99a4-e50de1a5d6a6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055579 Jul 20 15:22:15.142793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 775/2133] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:15 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:15.281995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:15.282992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:15.294192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.294949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.296259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.298983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.302979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:15.304816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:15.311167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.311601 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.312360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.312797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.316773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:15.397273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.405639 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-87a8def0-a224-4c41-a833-b7e8df24d5c5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.109391 Jul 20 15:22:15.406052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 776/2134] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:15.410423 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:15.411024 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:15.417079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.417530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.418292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.418722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.470468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.480049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-274b5c64-8cfb-4d8f-a8db-6cbf93a81710 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.204357 Jul 20 15:22:15.480769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1359/2135] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:15 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:15.485113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:15.485719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:15.492177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.492628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.493365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:15.493802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:15.496923 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.504960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8486c729-96ee-4929-ac46-0632268eb0f7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097996 Jul 20 15:22:15.505708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 777/2136] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:15 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:15.627681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.636645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.638131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:15.639314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-50d14762-576e-4d0e-b73d-535314013b4f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.157733 Jul 20 15:22:15.639896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1360/2137] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:16.159308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:16.160940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:16.172039 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.172779 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.174006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.174732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.210470 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d8a7e1f4-f14e-4fec-a743-b4d635917592 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058099 Jul 20 15:22:16.211021 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 778/2138] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:16.425931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:16.427019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:16.438166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.438916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.440242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.440975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.497604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:16.498302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:16.504668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.505113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.505873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.506297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.772719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:16.776623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:16.781827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a4ba00f9-eb55-42bd-adfa-842740a91284 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.362532 Jul 20 15:22:16.782411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1361/2139] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:16.793125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:16.793730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:16.799983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.800431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.801188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.801619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.872310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:16.880437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d68dde0d-373b-4ba8-99f8-ec170c739d76 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097060 Jul 20 15:22:16.881020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1362/2140] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:16.883807 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:16.885986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:16.886377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:16.892630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0a2410c8-c2b5-45f1-8eae-bf30b1dda916 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.398958 Jul 20 15:22:16.892833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.893065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:16.893178 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 779/2141] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:16 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:16.893813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:16.894239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.026336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:17.035256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:17.036789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:17.037963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-78c67f31-509d-4b73-a4ad-85ee6a055804 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.156041 Jul 20 15:22:17.038520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1363/2142] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:17.226649 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:17.227801 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:17.236231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.236826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.238008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.238647 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.275040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-09aed8d9-1527-4478-be75-0e19931bc913 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054955 Jul 20 15:22:17.275550 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 780/2143] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:17 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:17.802942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:17.803999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:17.815031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.815821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.817065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.817771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.880551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:17.890906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c49f05d0-6c43-4b15-a235-5c77de03873b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.093950 Jul 20 15:22:17.893411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1364/2144] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:17 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:17.898320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:17.898969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:17.905440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.905899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.906651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.907085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.909313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:17.909911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:17.916097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.916537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.917285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:17.917705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:17.976624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:17.984771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-31b32042-614c-47ac-93e3-67f926b69a36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090334 Jul 20 15:22:17.985304 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 781/2145] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:18.057019 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:18.057569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:18.062170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:18.063967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.064422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.065190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.065628 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.070394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5b7ea656-115e-4f26-94bc-e83408f3ea38 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164607 Jul 20 15:22:18.071040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1365/2146] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:17 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:18.210428 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:18.219499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:18.220973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:18.222127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3fd3ad66-1829-4016-b177-14e733af687e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.169121 Jul 20 15:22:18.222675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 782/2147] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:18.291167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:18.292941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:18.303125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.303898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.305056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.305486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.336534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4811f2c7-b843-4760-b078-011730c58ae6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052114 Jul 20 15:22:18.336978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1366/2148] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:18.911191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:18.911826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:18.920012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.920759 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.921981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:18.922722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:18.986445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:18.995205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-773fe834-a08a-4285-991c-4574186cd149 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087833 Jul 20 15:22:18.995989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 783/2149] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:18 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:19.002764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:19.003520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:19.010794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.011279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.012052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.012490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.077884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:19.086412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0860d0c2-e29d-43d3-889e-a648915e645e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088183 Jul 20 15:22:19.087445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1367/2150] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:19.089933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:19.089933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:19.094356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.094800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.095579 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.096007 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.234291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:19.239990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:19.240599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:19.243261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f568f07f-ac69-49b5-b629-e1cf79087fc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.159365 Jul 20 15:22:19.243842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 784/2151] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:19 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:19.247576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.248035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.248833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.249267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.349453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:19.350448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:19.357151 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.357611 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.358404 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:19.358838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:19.386270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:19.388575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bbd65346-b8ac-43fe-89ca-aef707d2ae11 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043054 Jul 20 15:22:19.389134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 785/2152] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:19.394918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:19.396535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:19.397707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-930fe631-e417-4bb7-9065-2096490edb7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161596 Jul 20 15:22:19.398261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1368/2153] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:22:20.015248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:20.016072 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:20.025334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.025925 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.026926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.027535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.096324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.104210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:20.104708 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-be68634e-2dc2-4d01-80ad-94ac95175962 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094611 Jul 20 15:22:20.104843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:20.105198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 786/2154] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:20 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:20.111491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.111935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.112691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.113122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.176440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.185390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e8c59457-5750-47b9-985a-ee42d0f5700f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084966 Jul 20 15:22:20.185956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1369/2155] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:20.261149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:20.261750 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:20.268378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.268777 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.269523 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.269948 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.400735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:20.401720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:20.408103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.408522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.409288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.409718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.410539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.418706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4d1fe6fc-50c7-4520-9bb0-5fcf0983cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161721 Jul 20 15:22:20.419341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 787/2156] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:20 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:20.423748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:20.424344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:20.430881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.431363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.432109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:20.432540 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:20.440492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9098f275-cb8d-4372-a9fa-cd6d57c4b296 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043921 Jul 20 15:22:20.441222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1370/2157] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:20.573854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.586267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.587804 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:20.588980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-20ce08cd-aa1a-474d-bd69-7a10d7630fd9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168721 Jul 20 15:22:20.589531 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 788/2158] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:21.124441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:21.125291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:21.135198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.135767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.136696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.137227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.202770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.204475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:21.205088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:21.211443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bb792d93-8817-4c91-8211-8268bd2d7fd7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092040 Jul 20 15:22:21.212034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1371/2159] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:21 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:21.212270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.212930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.213727 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.214160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.278073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.289811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-32772496-5501-487b-8ae3-68da5fc552b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088924 Jul 20 15:22:21.290426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 789/2160] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:21 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:21.439875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:21.440895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:21.449063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.449503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.450256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.450678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.452279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:21.453311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:21.459703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.460153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.460924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.461354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.492969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-606c2ce4-8b30-4aac-8573-cfa87585552d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044521 Jul 20 15:22:21.493547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 790/2161] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:21 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:21.589404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.599256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef331ed4-88e7-44ef-8a2f-51c5bf7943a1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.165343 Jul 20 15:22:21.600043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1372/2162] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:21 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:21.607248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:21.607854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:21.614006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.614444 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.615231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:21.615969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:21.760049 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.768882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.770412 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:21.771616 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-33af16d6-79c6-46d8-9c5b-28bea07091d4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168484 Jul 20 15:22:21.782247 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 791/2163] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:22.232810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:22.233402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:22.244916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.245366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.246149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.246582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.307340 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:22.308032 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:22.315011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.315859 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.316465 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.316947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.318227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.326937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c820a4c-4b93-4590-a590-6a511baac595 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.100334 Jul 20 15:22:22.327552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1373/2164] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:22 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:22.381676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.385157 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:22.390755 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7673a64f-9354-4977-8005-5b5750a26300 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087534 Jul 20 15:22:22.391347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 792/2165] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:22.508590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:22.509595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:22.516100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.516553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.517307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.517738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.550308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7e5f8dfa-d36c-4e1f-85ab-c0dcd3501d4e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048080 Jul 20 15:22:22.550886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1374/2166] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:22 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:22.619344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:22.619957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:22.626427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.626879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.627677 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.628112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.765040 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.773427 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3021d57c-0234-487b-89e8-c1d4420e1eff tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.159486 Jul 20 15:22:22.773988 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 793/2167] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:22 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:22.789829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:22.790437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:22.796616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.797060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.797799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:22.798225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:22.930152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.939397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.940986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:22.942152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8bba6b00-af86-4079-881b-dc580cd46c26 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.156177 Jul 20 15:22:22.942682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1375/2168] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:23.344235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:23.345380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:23.353145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.353596 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.354443 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.354961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.408005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:23.408638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:23.415216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.415656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.416413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.416848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.422125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:23.430124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3092fa65-8ce3-47fe-972f-e558c4403a1c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089853 Jul 20 15:22:23.430701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 794/2169] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:23.478650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:23.486861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9e661fad-708d-48ce-bec4-3d7021b0d0a4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082626 Jul 20 15:22:23.487411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1376/2170] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:23.566332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:23.567966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:23.578753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.579446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.584658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.587798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.622219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7fc72549-0e50-419a-82a9-91d3e88550f2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062500 Jul 20 15:22:23.622771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 795/2171] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:23.789993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:23.790628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:23.797026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.797479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.798239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.798677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.933998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:23.942443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27bd7ade-bcc3-4eca-a1aa-76da11be16e3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.156346 Jul 20 15:22:23.943023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1377/2172] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:23 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:23.961478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:23.962089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:23.968430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.968879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:23.969642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:23.970073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.103011 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:24.111624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:24.113030 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:24.114215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f467616c-8571-4ed9-8293-ef9c545ba19f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.156705 Jul 20 15:22:24.114739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 796/2173] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:24.449579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:24.450171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:24.457996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.458495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.459375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.459826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.479956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:24.504448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:24.505276 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:24.512330 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.512778 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.513556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.513990 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.525359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:24.533632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-70e411e3-4e12-448d-a0f3-dabe1aad7d64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088880 Jul 20 15:22:24.534207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1378/2174] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:24.574932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:24.582811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fa63e99b-72d3-4759-8e8f-f94206262a11 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082241 Jul 20 15:22:24.583393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 797/2175] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:24.634402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:24.635738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:24.644081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.644701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.645727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.646327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.680119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-419d0a39-96d4-4d2a-8092-146ec7be7fb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050166 Jul 20 15:22:24.680665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1379/2176] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:24.962327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:24.963192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:24.969460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.969905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:24.970654 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:24.971152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.101720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.114234 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5c839a0f-f9d2-4325-a286-8fc5d80036ee tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.157532 Jul 20 15:22:25.114969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 798/2177] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:24 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:25.132790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:25.133436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:25.139951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.140392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.141134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.141556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.278353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.287558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.289095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.290354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c34fceb4-ff0a-4af0-ab3a-c12cd106498c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161460 Jul 20 15:22:25.290915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1380/2178] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:25.555200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:25.556296 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:25.567568 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.568315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.569577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.570318 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.604616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:25.605249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:25.612141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.612603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.613381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.613818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.632549 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.640796 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b323da5a-321d-494f-a858-848bcd55785b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092114 Jul 20 15:22:25.641605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 799/2179] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:25.676683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:25.691102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c9908f5-1814-4f5a-a0dc-202f57a8d5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090261 Jul 20 15:22:25.691718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1381/2180] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:25.691991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:25.693022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:25.699333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.699784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.700541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:25.700973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:25.730405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0e828fa1-c059-463e-9070-30f9a2a386c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042285 Jul 20 15:22:25.730955 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 800/2181] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:22:26.136026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:26.136790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:26.144780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.145336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.146270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.146806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.289133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.297934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-acdfdf53-353f-4b35-93f7-80cc81cfab2f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167552 Jul 20 15:22:26.298509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1382/2182] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:26 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:26.316817 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:26.317430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:26.327104 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.327780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.328963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.329559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.465478 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.477354 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.478842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.480152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d5c95a18-468f-45c4-9d76-c7baa6a1ebc9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167262 Jul 20 15:22:26.480918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 801/2183] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:26.657862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:26.658479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:26.665384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.665747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.666497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.666924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.708235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:26.708822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:26.717380 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.717913 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.718786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.719320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.731386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.739808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc329c3f-34e9-4e09-9ef1-27c753fcc325 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085783 Jul 20 15:22:26.740383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1383/2184] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:26.745461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:26.746481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:26.752428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.752866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.753620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:26.754039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:26.784066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7dedb91a-c1cc-4f58-a5e5-eb657b7d12bd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042311 Jul 20 15:22:26.784638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1384/2185] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:26.784854 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:26.793031 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1a5fdfc3-50c4-4669-aaec-2e79b1631315 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088648 Jul 20 15:22:26.793594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 802/2186] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:27.321663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:27.322660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:27.332915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.333660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.334897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.335778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.477063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.485665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9e7096c5-f5c5-4176-8bd0-0e42a39796c6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.170521 Jul 20 15:22:27.486317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1385/2187] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:27 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:27.502185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:27.502787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:27.509273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.509712 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.510452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.510881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.650907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.663899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.665365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.666533 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-41a82a23-b3e0-44be-8d9e-b678f91b9d7a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168266 Jul 20 15:22:27.667142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 803/2188] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:27.758752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:27.759938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:27.767532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.767975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.768722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.769150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.796155 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:27.797067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:27.803891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.804332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.805087 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.805516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.837101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b73b6b0a-7e52-4af9-8fd3-6a5cb2f78cac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044681 Jul 20 15:22:27.837690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 804/2189] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:27 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:27.842037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:27.842573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.842735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:27.848545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.848989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.849734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:27.850160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:27.850665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-52ac365f-a141-49d7-8827-d155eb7bb1ac tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.097786 Jul 20 15:22:27.851270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1386/2190] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:27 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:27.910345 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:27.918476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3d8473cc-0c5f-43b7-b5b3-1e246987fbff tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.079864 Jul 20 15:22:27.919037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 805/2191] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:27 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:28.505162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:28.505966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:28.512812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.513257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.514007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.514438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.646219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:28.654861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b3b84052-1ce8-4f37-a541-be67aaf7b19e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.154547 Jul 20 15:22:28.655539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1387/2192] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:28 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:28.684769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:28.685378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:28.691788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.692236 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.693020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.693449 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.834395 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:28.843215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:28.844905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:28.846055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-de60fee6-4a7d-422e-a05f-fa9170042afe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165295 Jul 20 15:22:28.846606 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 806/2193] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:28.848732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:28.849671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:28.856387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.856841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.857601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.858038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.867284 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:28.867897 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:28.874134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.874588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.875373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.875819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.888791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6116c5fd-0542-4f25-88b7-6a0ccfead044 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044031 Jul 20 15:22:28.889359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1388/2194] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:28.936052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:28.936689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:28.937511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:28.943080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.943540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.944291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:28.944720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:28.945768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2d9d7948-1242-45f4-a33a-ccf96001da7a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082654 Jul 20 15:22:28.946322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 807/2195] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:29.009029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:29.017199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80aa6b9e-aff7-4105-83a4-c37a0626d38a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085067 Jul 20 15:22:29.017786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1389/2196] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:28 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:29.677006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:29.677863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:29.684993 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.685439 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.686193 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.686624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.826273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:29.834656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d5ca2299-8caa-443e-ae76-772a7c1a10dc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164351 Jul 20 15:22:29.835245 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 808/2197] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:29 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:29.864432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:29.865040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:29.871914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.872375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.873129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.873557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.900567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:29.901548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:29.909619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.910067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.910858 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.911322 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.928451 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:29.942663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1af38808-bc63-4b09-9bdf-595bf8efdca1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045964 Jul 20 15:22:29.943262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 809/2198] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:29.961513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:29.962119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:29.968210 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.968653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:29.969658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:29.970272 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.003234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:30.012170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:30.013674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:30.014821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27bb5da7-01be-4b88-94dd-d339a368edae tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.154335 Jul 20 15:22:30.015459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1390/2199] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:30.032295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:30.038179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:30.038926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:30.040775 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e7c44c2a-5fe4-4df4-8340-dda594b3562d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082948 Jul 20 15:22:30.041344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 810/2200] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:30.045284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.045737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.046532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.046964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.113214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:30.121817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-41085db6-964f-44c3-ad6b-f7d549f2c7da tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087332 Jul 20 15:22:30.122388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1391/2201] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:30.856160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:30.857273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:30.870526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.870977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.871780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.872215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.954400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:30.955456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:30.962272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.962720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.963538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:30.963974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:30.994780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-faf5cfa6-ab63-481a-8441-de71d189f683 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044421 Jul 20 15:22:30.998604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1392/2202] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:31.002502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.010734 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd906924-c8c7-479b-af6b-6ae1ee5a7b5b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161370 Jul 20 15:22:31.011315 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 811/2203] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:30 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:31.033329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:31.033932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:31.040378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.040821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.041579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.042011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.057788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:31.058499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:31.064875 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.065326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.066091 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.066525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.140721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.149899 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8df26a03-8cf0-4cbd-b84d-db253f6297c7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096007 Jul 20 15:22:31.150489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 812/2204] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:31.154952 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:31.155554 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:31.161942 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.162367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.163169 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:31.163648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:31.202537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.211344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.212854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.214034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-94c572fc-79b0-4cbe-b22f-b5f7a2c48651 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.184594 Jul 20 15:22:31.214607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1393/2205] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:31.230271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:31.238522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-abf7201b-65d3-44d8-b87a-e3deacf4be96 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087093 Jul 20 15:22:31.239065 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 813/2206] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:32.008084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:32.009023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:32.016064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.016510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.017257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.017662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.028044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:32.028658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:32.035693 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.036051 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.036817 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.037248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.048398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-098c3563-5d54-4de2-98d1-afbf1b3db371 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045585 Jul 20 15:22:32.048966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1394/2207] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:32 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:32.167660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:32.168341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:32.175398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.175854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.176603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.177029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.177625 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.181310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:32.186326 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3345938e-baee-4b4e-90ca-17cac3ccfdeb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.162230 Jul 20 15:22:32.186886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 814/2208] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:32 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:32.231857 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:32.232491 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:32.238883 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.239366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.240218 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.241001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.242638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.251091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da8fea25-8c90-4224-b502-80cdb25067e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087488 Jul 20 15:22:32.257039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1395/2209] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:32.263214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:32.263839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:32.265672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:32.271405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.271855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.272617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:32.273032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:32.339421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.347583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-832ca031-1d58-42fb-a7b2-0fa36337c27e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089009 Jul 20 15:22:32.348151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1396/2210] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:32.401840 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.410420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.411949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:32.413102 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5480aeb8-2c14-4abd-aade-652802127ccb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.185264 Jul 20 15:22:32.413700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 815/2211] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:33.064428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:33.065930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:33.074382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.074830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.075619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.076054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.105279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cad76860-7a3b-400b-949a-345e043d009e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047484 Jul 20 15:22:33.105824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1397/2212] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:33.207355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:33.208344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:33.216261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.216703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.217448 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.217866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.271340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:33.272268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:33.279087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.279540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.280301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.280739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.353394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.361547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-637c588b-c1fc-4cc1-b14e-9fe5633a0978 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096499 Jul 20 15:22:33.362119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1398/2213] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:33 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:33.369811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:33.371454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:33.377863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.378450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.379710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.379942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.385962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.396917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2f723c9e-e670-4532-91ef-24b387eb9264 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.195858 Jul 20 15:22:33.397512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 816/2214] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:33 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:33.430739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:33.431365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:33.433183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:33.438665 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.439189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.439966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:33.440377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:33.442952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.451125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a38c2aa1-39ca-4afa-95df-d48732c63a1f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086420 Jul 20 15:22:33.451690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1399/2215] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:33.569967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.578950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.580499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:33.581725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b067ea35-6eb7-49ed-98d0-a3d553cd6df1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.154829 Jul 20 15:22:33.582205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 817/2216] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:22:34.120717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:34.121797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:34.129714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.130245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.131172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.131693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.164496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-64f95ef3-e849-40ba-a98a-6a8dfce81b90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049398 Jul 20 15:22:34.165086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1400/2217] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:34 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:34.382893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:34.383527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:34.390767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.391253 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.392012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.392456 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.414244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:34.414864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:34.421562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.422067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.422826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.423384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.454414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.463243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-66f6ce4f-db97-4702-85e7-953727a91a96 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086453 Jul 20 15:22:34.463780 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 818/2218] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:34.468262 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:34.468880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:34.475191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.475651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.476493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.476924 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.547884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.556906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5e53f209-14ce-4b3d-bf82-9dc1025e1bf3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092245 Jul 20 15:22:34.557704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 819/2219] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:34.581422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.590058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0a5458ae-250e-471d-958c-8e24cf0f89d5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.179949 Jul 20 15:22:34.590591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1401/2220] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:34 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:34.600299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:34.600994 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:34.607581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.608028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.608786 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:34.609216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:34.738949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.748090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.749567 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:34.750722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2f5e4c3e-50ae-42f4-a4ad-614bd90e28a0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.154355 Jul 20 15:22:34.751309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 820/2221] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:35.178630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:35.180529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:35.190122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.190768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.191928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.192544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.227100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-950800eb-9661-42b6-abfa-407101a70468 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054000 Jul 20 15:22:35.227991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1402/2222] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:35 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:35.485651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:35.486662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:35.495985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.496476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.497312 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.497795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.561922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.573860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-02151ad2-8605-429f-bd15-c653f5b7ac0a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.095099 Jul 20 15:22:35.574622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 821/2223] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:35.576517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:35.577362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:35.586458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.586927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.587804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.588230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.608038 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:35.608680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:35.615078 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.615876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.616690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.617127 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.651559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.660019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c3c61cc6-d1b0-4ef6-9f37-7c9ffb5ba716 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089261 Jul 20 15:22:35.660587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1403/2224] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:35.751889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.760534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8518975a-b5da-431f-a33e-538349d89dbe tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.156266 Jul 20 15:22:35.761092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 822/2225] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:35 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:35.769264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:35.769919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:35.776631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.777072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.777954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:35.778393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:35.911078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.920472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.922151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:35.923462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0132eba9-6b89-4993-83fa-f03b67529d00 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.158116 Jul 20 15:22:35.924096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1404/2226] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:36.244787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:36.245928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:36.253069 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.253570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.254415 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.254890 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.287827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-23133022-01af-4066-93da-1369ad88fff6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048810 Jul 20 15:22:36.288403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 823/2227] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:36.597715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:36.598712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:36.606754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.607230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.607974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.608397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.676512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:36.677099 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:36.681599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:36.684041 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.684482 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.685230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.685664 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.689782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-01761ea2-0736-4ba8-8f97-1155763c298b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.098407 Jul 20 15:22:36.690356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1405/2228] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:36 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:36.749401 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:36.757496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-64b53f86-f608-44a3-85c0-dcbd78f86018 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084820 Jul 20 15:22:36.758035 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 824/2229] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:22:36.777789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:36.778480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:36.784806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.785250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.785998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.786550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.918677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:36.930219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02a4f3f6-ec5d-48d1-8d5d-f56d25b34893 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.156212 Jul 20 15:22:36.930776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1406/2230] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:36 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:36.943085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:36.943701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:36.950229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.950692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:36.951496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:36.951932 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.094946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:37.103887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:37.105426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:37.106608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-77ed8e46-960b-4ca7-9e6e-30fdf93330e3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167396 Jul 20 15:22:37.107156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 825/2231] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:37.300299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:37.301457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:37.308222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.308715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.309484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.309928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.340427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e1e1dc7-c66b-4f18-89f6-e30eaaa1ea48 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044265 Jul 20 15:22:37.340987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1407/2232] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:37 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:37.707183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:37.707731 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:37.714685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.715176 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.715944 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.716374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.720362 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:37.775313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:37.775932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:37.778818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:37.782774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.783253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.784019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.784457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.787008 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-803676b6-60d3-4bd0-a9d4-20ec93b2a695 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.083841 Jul 20 15:22:37.787609 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 826/2233] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:37 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:37.848467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:37.856579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4c87c99d-b7f1-443a-a13c-e3d746ed38aa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086055 Jul 20 15:22:37.857142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1408/2234] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:37 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:37.948746 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:37.949377 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:37.956216 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.956668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:37.957509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:37.957874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.091972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.100405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d82d2ac5-a0bb-41ee-a736-14bb0f1567bc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.155986 Jul 20 15:22:38.101067 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 827/2235] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:37 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:38.125664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:38.126324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:38.132550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.133001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.133754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.134184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.275746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.288721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.290205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.291393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9f3a6fc6-ccfc-4432-88cd-a340d9d059a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.169572 Jul 20 15:22:38.291925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1409/2236] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:38.357358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:38.359300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:38.369674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.370180 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.370964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.371447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.402822 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0be2afa6-d57c-48a5-8237-154efef917d2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052869 Jul 20 15:22:38.403413 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 828/2237] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:38.808675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:38.810476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:38.820194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.820645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.821405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.821840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.874202 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:38.874787 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:38.881729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.882308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.883082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:38.883560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:38.889186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.897593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d61a4664-1fcb-4624-855b-11ecce5e0f59 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096628 Jul 20 15:22:38.898187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1410/2238] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:38 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:38.945588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:38.953884 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-665319b8-69b8-4124-9a1e-4294d060c235 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083553 Jul 20 15:22:38.954450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 829/2239] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:39.119926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:39.120470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:39.127269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.127731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.128514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.128963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.268926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:39.272324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:39.277777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fb9f6d9b-5b3e-4c95-8667-30d72d53cc57 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163457 Jul 20 15:22:39.278347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1411/2240] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:39 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:39.315914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:39.316657 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:39.322843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.323332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.324129 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.324564 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.414457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:39.415586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:39.422640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.423077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.423898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.424318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.452955 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:39.454758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6fed2bc4-b0ae-4aed-a1bb-c71bde82f75e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044685 Jul 20 15:22:39.455403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1412/2241] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:39.464004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:39.466028 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:39.467920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2d53bf65-41e8-48e8-b5cc-4db4f081eb74 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.155627 Jul 20 15:22:39.468651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 830/2242] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:39.914610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:39.915300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:39.922394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.922840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.923626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.924056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.971074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:39.971645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:39.978506 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.978958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.979767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:39.980203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:39.991282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.000403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9014bc8-accb-4521-bb22-a2dfbad0aace tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089685 Jul 20 15:22:40.001004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1413/2243] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:40.044762 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.053147 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2fd932fd-0bc7-47a0-8e6d-df9cfd977ce5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086092 Jul 20 15:22:40.053715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 831/2244] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:40.299308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:40.300313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:40.310839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.311635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.312876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.313591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.450043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.459688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0fb4667f-3da7-4f4a-80a3-30d25f775b39 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166931 Jul 20 15:22:40.460230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1414/2245] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:40 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:40.465929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:40.466843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:40.473235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.473676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.474421 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.474844 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.488718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:40.489396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:40.496319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.496767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.497525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:40.497923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:40.504705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8bf8a323-a612-4a61-80c6-5dcc5aeecdf2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042621 Jul 20 15:22:40.505254 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 832/2246] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:40 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:22:40.636077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.644691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.646216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:40.647331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ac39e98d-fd23-407b-9e6c-cd0c1a263084 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.162815 Jul 20 15:22:40.648010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1415/2247] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:41.022075 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:41.022991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:41.033944 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.034698 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.036018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.036757 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.069903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:41.070644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:41.077737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.078266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.079148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.079657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.099685 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.108372 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a3240b7d-d77a-46dc-87f8-a963988c53ba tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092965 Jul 20 15:22:41.108944 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 833/2248] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:41 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:41.141840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.150244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff800296-2ee6-49f7-acc8-e75699fd3a20 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084265 Jul 20 15:22:41.150813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1416/2249] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:41.480614 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:41.481623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:41.492460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.492906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.493629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.494119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.516726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:41.517714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:41.524191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.524645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.525408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.525848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.555816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d5d41eb-897e-42fc-80f7-f52c7dbeadc6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042995 Jul 20 15:22:41.556591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1417/2250] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:41.628847 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.637690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c2974103-b668-4c07-8dbf-ac7ecf4ca65b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.163547 Jul 20 15:22:41.638250 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 834/2251] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:41 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:41.666084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:41.666700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:41.673153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.673601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.674362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:41.674794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:41.816969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.826357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.827896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:41.829081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-67ccf3d2-72c6-4907-b735-d721a41f3ec5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.166896 Jul 20 15:22:41.829643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1418/2252] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:42.129824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:42.130784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:42.142215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.142965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.144283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.145000 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.167517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:42.168210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:42.175613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.176059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.176822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.177224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.208097 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:42.216283 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c046e082-96ea-498c-8200-8e8a48515423 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092944 Jul 20 15:22:42.216827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 835/2253] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:42.238525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:42.247238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2fc9770d-19e9-41a4-b95f-3e2bb0f22ef1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083662 Jul 20 15:22:42.247778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1419/2254] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:42.572450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:42.574073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:42.584715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.585570 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.586892 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.587666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.629369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8e064833-bb3c-4c67-890e-9c58d17beb9f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063944 Jul 20 15:22:42.630059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 836/2255] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:42 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:42.654534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:42.655164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:42.661571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.662014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.662756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.663311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.797731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:42.806381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-04aaef76-63e7-4b85-89da-f2f3039a68fb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.155771 Jul 20 15:22:42.806939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1420/2256] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:42 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:42.859908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:42.860919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:42.869464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.869917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:42.870706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:42.871158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.002536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.011146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.012634 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.013799 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-08fcb878-8340-41c8-a23c-5f63f6a82291 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161715 Jul 20 15:22:43.014363 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 837/2257] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:43.237504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:43.238515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:43.250215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.250903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.252215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.252929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.264328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:43.264958 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:43.272095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.272544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.273360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.273908 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.323014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.331371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-17db1c30-f9ae-4ed1-8369-406763b0320f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.100332 Jul 20 15:22:43.331952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1421/2258] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:43.353515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.362133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8ca07e56-00d6-4c91-a61f-5517c125781e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.101632 Jul 20 15:22:43.362700 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 838/2259] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:43.646153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:43.647891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:43.654397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.654848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.655644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.656083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.687721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-10cd9af3-0ea1-40b4-b2d8-42d2b631705e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048376 Jul 20 15:22:43.688321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1422/2260] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:43 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:43.823763 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:43.824405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:43.830850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.831338 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.832103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:43.832686 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:43.969874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:43.978573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-494f9e8c-285c-480d-80e8-225186c84ad0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.159011 Jul 20 15:22:43.979137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 839/2261] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:43 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:44.035789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:44.036772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:44.043828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.044303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.045114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.045564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.191564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:44.200965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:44.202489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:44.203691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f672e6e6-4195-4769-a105-e62cdb4a0df7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.174773 Jul 20 15:22:44.204275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1423/2262] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:44.352690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:44.353646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:44.364832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.365981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.366983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.367447 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.379676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:44.380369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:44.387315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.387767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.388526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.388957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.439630 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:44.447946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-87410e94-c796-449e-a55f-56799e697a87 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.101816 Jul 20 15:22:44.448532 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 840/2263] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:44 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:44.456765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:44.466561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7240cdcc-b1aa-41a6-a7bc-9d5f48037117 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090721 Jul 20 15:22:44.467193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1424/2264] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:44.704243 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:44.705709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:44.716133 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.716881 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.718140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:44.718827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:44.751842 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d77cf112-29ba-42ea-afc0-f276f828bdba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054476 Jul 20 15:22:44.752394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 841/2265] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:44.999223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:44.999917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:45.009001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.009474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.010299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.010758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.152859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.161306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b83d298a-8dea-411b-9c1c-32e5ba5fdf0a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168366 Jul 20 15:22:45.161849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1425/2266] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:44 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:45.231729 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:45.232653 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:45.242090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.242536 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.243364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.243802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.380235 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.383871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:45.389976 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.391545 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.392710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-15051d07-8c08-42c3-be37-5b4c452515dd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167029 Jul 20 15:22:45.393264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 842/2267] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:45.469847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:45.470804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:45.480023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.480464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.481228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.481651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.484146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:45.484728 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:45.491835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.492278 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.493086 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.493527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.566150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.567833 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:45.574437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef7ecfad-b252-4d22-afd0-ad9321ea9b90 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.110994 Jul 20 15:22:45.575124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1426/2268] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:45.575966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fbb96871-9958-4a76-bf39-f6908abc0167 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.096012 Jul 20 15:22:45.576527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 843/2269] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:45.767334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:45.768987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:45.779329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.780076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.781329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:45.782000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:45.813801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d7cad3f-e5c9-4384-a144-8e8c63d013cd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053137 Jul 20 15:22:45.814349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1427/2270] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:46.181641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:46.182273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:46.189269 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.189753 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.190548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.191001 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.333374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.342525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2893ac79-07f4-4083-b62f-3e8f8a31f892 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.166365 Jul 20 15:22:46.343232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 844/2271] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:46 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.413682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:46.414361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:46.421516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.422043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.422886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.423405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.444132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:46.564985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.576726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.579000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.580797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ccffc8d1-bd51-4698-8e39-5fb895d25755 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.172556 Jul 20 15:22:46.581500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1428/2272] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:46.591539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:46.592203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:46.592511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:46.593122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:46.599090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.599249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.599547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.599713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.600311 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.600473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.600782 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.600942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.685813 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.687926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:46.694496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0c3c9c3a-7f99-4b30-8d4c-d4b0aebd6d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.106527 Jul 20 15:22:46.695023 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 845/2273] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:46 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.696522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a44ce9b-7720-435a-8d88-67a5691183d4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.107616 Jul 20 15:22:46.697130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1429/2274] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:46.832256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:46.834079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:46.841177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.841626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.844852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:46.845344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:46.881711 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4da20fbe-6354-43bb-b1b0-7fad8d9baba3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056633 Jul 20 15:22:46.882335 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 846/2275] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:46 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:47.363936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:47.365432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:47.372563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.373061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.373822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.374259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.519160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.527361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-88a4c306-6052-461b-9849-7783cf4c7bc1 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169792 Jul 20 15:22:47.527940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1430/2276] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:47 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:47.603559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:47.604457 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:47.615930 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.616705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.617477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.617915 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.713780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:47.714417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:47.723787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.724258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.725010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.725448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.777460 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.786022 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.787595 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.788926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ec88962d-7e1e-43fb-9007-98946a3bb9c5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.192066 Jul 20 15:22:47.789683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 847/2277] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:47.794136 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:47.794732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:47.795597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.801225 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.801699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.802502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.802985 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.803957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f971ab68-3463-4984-a993-7c00d1cc3bb2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.095672 Jul 20 15:22:47.804532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1431/2278] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:47 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:47.867604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:47.879494 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-4e1c8d1d-acd1-4785-94ea-c1623eb4ba37 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088469 Jul 20 15:22:47.880149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 848/2279] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:47.896008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:47.897224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:47.903762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.904326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.905506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:47.906120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:47.938962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b447438d-b601-4ce3-88e7-b6caa29c99a2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049158 Jul 20 15:22:47.942857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1432/2280] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:48.552748 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:48.553676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:48.563701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.564435 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.565577 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.566230 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.703879 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:48.716735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c98b3169-ca76-44f3-91f6-2f2cd6398f6e tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.169534 Jul 20 15:22:48.717627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 849/2281] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:48 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:48.809000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:48.809616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:48.816485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.816970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.817730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.818168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.822293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:48.822947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:48.829426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.829920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.830740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.831267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.915036 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:48.924694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0790a2f7-9006-4014-8c4e-751c5ec1977e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.106577 Jul 20 15:22:48.925337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 850/2282] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:48 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:48.931004 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:48.931650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:48.938876 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.939375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.940183 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:48.940790 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:48.987859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:48.997002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:48.998489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:49.000029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-90e662e6-b4cc-4a99-a7ba-8e185e14376a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.195472 Jul 20 15:22:49.001023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1433/2283] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:49.005869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:49.006746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:49.011403 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:49.016392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.016778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:49.017544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.017975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:49.020847 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-04da7292-31af-4b75-b150-95e12ef1b316 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094556 Jul 20 15:22:49.021629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 851/2284] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:49.047512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eff76b4e-2af4-43a9-91b0-3da2750d7aa2 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045275 Jul 20 15:22:49.048065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1434/2285] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:49.741719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:49.742334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:49.749637 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.750113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:49.750914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.751413 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:49.893418 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:49.902107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-33c964b9-468d-4361-85a4-bcdbf050b3b0 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.167230 Jul 20 15:22:49.902682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 852/2286] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:49 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:49.944186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:49.944952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:49.951934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.952379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:49.953122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:49.953543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.019414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:50.019935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:50.025393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:50.026499 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.027020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.027871 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.028308 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.033592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4c6602b0-6e10-4711-aa79-8ffbd0d45c94 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.095366 Jul 20 15:22:50.034174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1435/2287] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:49 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:50.038690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:50.039313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:50.045434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.045873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.046627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.047066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.121549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:50.130076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b1ffad4a-3840-4f78-a2aa-b616b7ab6c70 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094930 Jul 20 15:22:50.139939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1436/2288] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:50.139939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:50.139939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:50.142497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.142947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.143735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.144169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.174232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d9dd93d-8dc2-495d-85ef-072c3e0bdb0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042521 Jul 20 15:22:50.174806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1437/2289] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:50.184430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:50.193045 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:50.194594 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:50.195798 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d405292-0736-4afa-b9f9-7ea65b0686a9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.180540 Jul 20 15:22:50.196353 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 853/2290] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:50.919092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:50.919695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:50.927896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.928441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:50.929627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:50.930253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.050781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:51.051558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:51.058461 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.058914 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.059699 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.060156 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.075040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.083273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-22c0f5ee-afdd-46d4-8496-21c2de734e69 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.168092 Jul 20 15:22:51.083854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1438/2291] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:50 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:51.127929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.136201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d0e3878e-822c-4872-81ef-b02ef737e61a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089372 Jul 20 15:22:51.136805 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 854/2292] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:51 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:51.147459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:51.148242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:51.154426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.154870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.155647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.156085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.186502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:51.187907 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:51.194764 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.195287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.196074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.196527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.234995 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-03203aa2-43a6-46a2-906f-321e6b061288 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052540 Jul 20 15:22:51.235636 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 855/2293] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:51.240906 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:51.241589 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:51.248508 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.248959 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.249781 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:51.250215 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:51.378727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.386867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3e0811d-6d23-4723-a4d8-41be76e161fa tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.243457 Jul 20 15:22:51.387447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1439/2294] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:51.407977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.416806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.418314 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:51.419510 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c18e1b48-3ff2-4b49-a2cc-1c67829bb20d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.182949 Jul 20 15:22:51.420061 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 856/2295] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:52.101394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:52.102104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:52.110677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.111355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.112290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.112834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.153054 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:52.153797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:52.160349 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.160784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.161544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.161981 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.234461 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.244189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ad8ccc41-abce-442f-a123-117a36418dd2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094791 Jul 20 15:22:52.244850 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 857/2296] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:52 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:52.251200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:52.252137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:52.259079 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.259530 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.260299 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.260937 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.268753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.280830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-054cef46-93dc-4167-9fdd-07255a443c05 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.184358 Jul 20 15:22:52.281677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1440/2297] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:52 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:52.291467 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1370837d-b0bd-4353-aace-2e12ca67fad7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044146 Jul 20 15:22:52.292026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 858/2298] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:52.408216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:52.409054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:52.416923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.417367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.418095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.418524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.437887 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:52.438555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:52.445370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.445860 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.446679 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:52.447196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:52.467551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:52.486628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.495848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ac14926b-1685-491c-995f-4404377460a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094059 Jul 20 15:22:52.496458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1441/2299] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:52.579258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.592429 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.593969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:52.595173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bd726af7-5bfe-47f0-9e42-39e45d867983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161224 Jul 20 15:22:52.595709 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 859/2300] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:53.262914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:53.263607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:53.271326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.271827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.272679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.273157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.298116 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:53.298820 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:53.306044 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.306526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.307370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.307818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.337029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.345222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c277b336-fe6f-4bdd-8702-2f374de43e4e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086621 Jul 20 15:22:53.345781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1442/2301] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:53 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:53.351033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:53.351987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:53.358014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.358467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.359276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.359715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.391134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2850b2ac-38c1-4585-80ae-9a2de72b36e9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043504 Jul 20 15:22:53.392103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1443/2302] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:53.443732 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.456135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e8023060-75ae-4171-acfd-d86e51c91a88 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.161769 Jul 20 15:22:53.456846 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 860/2303] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:53 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:53.513172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:53.513722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:53.520171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.520631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.521389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.521824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.581937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.590098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d8eeee53-725f-490c-905c-80a53dc42452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080873 Jul 20 15:22:53.590660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1444/2304] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:53 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:53.619605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:53.620217 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:53.626390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.626837 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.627626 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:53.628060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:53.760558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.769719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.771177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:53.772369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f36456b6-d268-4fb7-b025-1ffe5330e80b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.156723 Jul 20 15:22:53.772968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 861/2305] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:54.366638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:54.367304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:54.375003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.375530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.376396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.376879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.381270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.403604 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:54.404623 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:54.411929 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.412426 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.413173 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.413736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.444541 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c3762de5-1498-433c-b3e6-555b3211bbb3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044918 Jul 20 15:22:54.445138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 862/2306] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:54.446069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.456994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02405537-ae3b-476b-a298-d33ff0e6ba27 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096825 Jul 20 15:22:54.457743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1445/2307] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:54 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:54.474220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:54.474838 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:54.481274 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.481723 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.482483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.482910 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.607067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:54.607677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:54.609542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:54.614002 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.615753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.616268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.617010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.617401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.622381 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0bba52de-e4e0-4550-a7bf-2db4720e225b tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.151952 Jul 20 15:22:54.623074 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 863/2308] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:54 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:54.689733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.699799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9401943b-d0bb-49c8-956e-ea7f0c431347 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.096618 Jul 20 15:22:54.700363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1446/2309] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:54 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:54.796026 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:54.796666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:54.802987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.803703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.804521 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:54.804957 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:54.939889 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.948816 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.950327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:54.951795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-20f90e9d-6962-4a03-85bf-98ddf01a02e6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.159916 Jul 20 15:22:54.952384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 864/2310] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:55.459806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:55.461456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:55.471887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.472599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.473664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.474272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.476211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:55.476795 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:55.483905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.484342 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.485177 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.485547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.505536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-15519160-cdaa-4f77-bc6e-241f2017ca6f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052696 Jul 20 15:22:55.506071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1447/2311] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:55.547559 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:55.555926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-6c7dc85d-ce60-46a6-89c2-37d5ef4e0d25 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085166 Jul 20 15:22:55.556483 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 865/2312] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:55 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:55.643989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:55.644701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:55.652031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.652470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.653209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.653634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.723367 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:55.723979 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:55.730268 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.730715 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.731513 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.731949 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.808200 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:55.816619 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-cf53c66a-2de2-4458-ae5d-7cb1d94aa070 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097313 Jul 20 15:22:55.817199 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 866/2313] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:55.821630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:55.830388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72f6cc12-757b-4d34-ad35-290864b1b559 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.192822 Jul 20 15:22:55.830913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1448/2314] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:55 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:55.973690 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:55.974738 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:55.976953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:22:55.982826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.983305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:55.984058 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:55.984490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.389767 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:56.398455 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:56.400005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:56.401174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-a3e71133-07c0-4b19-ac64-52762467cdfa tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.434061 Jul 20 15:22:56.401745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 867/2315] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:56.517622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:56.518547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:56.525490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.525993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.526736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.527198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.564271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9e18d409-8a07-489a-b3ee-e703ab40bd18 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050633 Jul 20 15:22:56.564854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1449/2316] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:56 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:56.572853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:56.573469 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:56.580169 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.580680 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.581375 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.581806 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.645464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:56.653776 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-527d56d9-0924-46ed-a9ec-14911a5ec5a7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.084759 Jul 20 15:22:56.654355 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 868/2317] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:56 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:56.834118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:56.834657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:56.841465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.841914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.842656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.843080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.849320 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:56.849918 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: e827c810-220b-4fe1-9d16-01bc8372b933] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:56.856421 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.856868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.857629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:56.858060 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:56.914944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:56.923608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f814542f-df8d-44f6-96fe-8bf0930fc5f2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093418 Jul 20 15:22:56.924240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1450/2318] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:56 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:57.000869 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:57.009334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-eab456aa-0841-4e4e-be1a-6c0f313e3557 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933" status: 200 len: 1560 microversion: 2.1 time: 0.164287 Jul 20 15:22:57.009880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 869/2319] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:56 2022] GET /compute/v2.1/servers/e827c810-220b-4fe1-9d16-01bc8372b933 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:57.423607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:57.424551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:57.433000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.433438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.434183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.434603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.574161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:57.576411 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:57.577493 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:57.582901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:57.584409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:57.584582 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.585059 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.585577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ccb20006-e737-4e7f-bf32-0c80f452e1a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168515 Jul 20 15:22:57.585832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.586110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1451/2320] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:57 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:57.586271 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.621710 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2b97c849-e74a-4776-9336-48ad8c106508 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049356 Jul 20 15:22:57.622185 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 870/2321] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:57.675366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:57.676381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:57.686994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.687819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.689076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.689789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.757739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:57.766100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dafed84f-5435-4086-84a6-1f932469293b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.097529 Jul 20 15:22:57.766637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1452/2322] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:57 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:57.943650 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:57.944557 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:57.960561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.960561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:57.960561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:57.960561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.029964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:58.039773 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-369d8ea4-7780-4449-9656-fe1b8d38ffdb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.101020 Jul 20 15:22:58.040057 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 871/2323] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:58.613094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:58.614219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:58.622301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.622766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.623593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.624020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.633103 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:58.634128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:58.641188 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.641660 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.642477 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.642963 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.673878 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1e0ba819-2f4f-40b7-a48f-52a3b73ed698 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044894 Jul 20 15:22:58.676407 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 872/2324] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:58.755156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:58.764681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:58.766245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:58.767602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-29afbae9-c6df-41d1-9bb6-f3521912da4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161467 Jul 20 15:22:58.768408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1453/2325] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:22:58.788475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:58.789089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:58.795492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.795935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.796703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:58.797153 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:58.861396 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:58.870566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-90baf25b-9ced-4956-bb68-18742289d07f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086231 Jul 20 15:22:58.871141 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 873/2326] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:58 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:59.063788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:59.064669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:59.072436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.072883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.073639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.074071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.147561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:59.155894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44ec5cc2-e81b-4a92-8db1-0fb1dbdccf87 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.098127 Jul 20 15:22:59.156451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1454/2327] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:59.690406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:59.691864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:59.699894 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.700337 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.701095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.701524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.731852 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-afb5329b-e8b4-46e3-9b7f-d8c49f528686 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048304 Jul 20 15:22:59.732406 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 874/2328] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:22:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:22:59.787572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:59.788181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:59.795903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.796357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.797118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.797720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.888364 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:22:59.888940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:22:59.895356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.895788 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.896651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:22:59.897143 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:22:59.967324 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:59.970829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:59.975678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8ace20fb-ce40-483b-a5b1-9cf0b57ec71a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.090906 Jul 20 15:22:59.976006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 875/2329] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:22:59 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:22:59.980121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:59.981876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:22:59.983195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d6848795-8d51-4a0c-9255-0073ab7ca76f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.199623 Jul 20 15:22:59.983826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1455/2330] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:22:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:00.174525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:00.175332 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:00.182476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:00.182926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:00.183737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:00.184184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:00.188277 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:00.258270 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:00.266866 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-623e24ad-7436-4167-9f02-dfedf91b9c7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097222 Jul 20 15:23:00.267481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 876/2331] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:00.745908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:00.746887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:00.753352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:00.753792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:00.754534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:00.754959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:00.784080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-985474e2-feee-4311-9267-71d076232611 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042298 Jul 20 15:23:00.784623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1456/2332] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:00.994704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:00.995327 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:01.002471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.002950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.003754 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.004191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.008139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:01.008863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:01.015615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.016054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.016801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.017223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.076113 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:01.084292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ffa06fcd-98b1-4269-ac81-c1dbbb8174d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094450 Jul 20 15:23:01.085052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 877/2333] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:00 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:01.160964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:01.169692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:01.171200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:01.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea33366e-569d-4bbe-b8a7-79227ccad62a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168083 Jul 20 15:23:01.172902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1457/2334] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:01.288471 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:01.288997 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:01.295490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.295943 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.296703 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.297137 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.363873 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:01.372646 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-3922cfdc-a585-4a74-8d47-fa251091914f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090794 Jul 20 15:23:01.373213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 878/2335] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:01.800120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:01.801673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:01.809236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.809848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.810625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:01.811073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:01.840738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ddc3c115-d788-4422-a3dd-50e8d6ca7a83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047226 Jul 20 15:23:01.841278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1458/2336] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:02.106017 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:02.107010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:02.118419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.119629 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.120936 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.121565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.191061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:02.192047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:02.194066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:02.198891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.199440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.200304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.200805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.202503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2c6a9436-fa03-40ca-a495-0950fe70e0b5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.103348 Jul 20 15:23:02.203319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 879/2337] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:02 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:02.335781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:02.344621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:02.346115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:02.347325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-992b0749-a6b5-4a6d-aac6-cf18b2bd12f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.160200 Jul 20 15:23:02.347881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1459/2338] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:02.390436 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:02.390973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:02.397525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.397977 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.398741 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.399229 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.460148 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:02.468518 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0d7a6ea7-0739-4633-92e3-fb7c063894cf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081997 Jul 20 15:23:02.469083 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 880/2339] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:02.855836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:02.856768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:02.863338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.863789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.864833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:02.865272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:02.894879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-30bd920f-e0dc-4dc1-86c3-9a27042f2326 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044512 Jul 20 15:23:02.895457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1460/2340] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:02 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:03.223713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:03.224600 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:03.236005 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.236743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.237989 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.238704 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.313052 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:03.325422 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bef01289-89af-448d-864f-eff3380d651d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.108052 Jul 20 15:23:03.326241 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 881/2341] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:03 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:03.368228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:03.368834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:03.375770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.376206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.376954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.377379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.485954 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:03.486573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:03.493344 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.493785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.494561 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.494996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.515805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:03.519311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:03.524775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:03.526344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:03.527569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d5686cab-d004-4264-b19d-01164ecc3d4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165365 Jul 20 15:23:03.528168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1461/2342] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:03.556277 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:03.564560 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-d28d9590-975c-48ae-895e-cef690d1c98b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082528 Jul 20 15:23:03.565120 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 882/2343] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:03 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:03.754762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:23:03.755085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:23:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:23:03.911271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:03.912764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:03.923236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.923981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.925224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:03.925979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:03.955498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6134a7be-559f-4c38-b491-af27f301da38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051185 Jul 20 15:23:03.956041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1462/2344] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:04.350289 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:04.350996 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:04.358617 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.359158 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.360003 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.360485 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.430149 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:04.438333 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-71e1cdef-b8e3-49ee-8340-f2a02cc08cf7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094656 Jul 20 15:23:04.438893 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 883/2345] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:04 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:04.512027 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:23:04.512195 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:23:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:23:04.546294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:04.546903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:04.553946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.554392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.555170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.555607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.581848 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:04.582461 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:04.588905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.589360 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.590186 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.590658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.656819 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:04.664946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c8c17b29-bee9-4b40-b7ce-1615417a1d3c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086970 Jul 20 15:23:04.665481 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 884/2346] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:04.711079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:04.720638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:04.722108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:04.723344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-560c98ae-9b8d-4e90-8d4d-0ad42f45aaa5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.180984 Jul 20 15:23:04.723903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1463/2347] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:04.970484 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:04.971658 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:04.979163 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.979534 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:04.980319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:04.980743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.010085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-66f27105-4c87-4964-9c51-09d759c07407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046105 Jul 20 15:23:05.010645 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 885/2348] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:05.459603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:05.460305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:05.467431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.467928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.468763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.469240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.534226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:05.542573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d94eba3b-c855-41d7-b029-8559a54ac0ea tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089596 Jul 20 15:23:05.543166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1464/2349] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:05 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:05.684662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:05.685453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:05.694288 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.694735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.695522 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.695945 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.741585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:05.742290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:05.749055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.749588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.750431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:05.750933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:05.768446 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:05.776691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-71b8e041-bb01-4067-b8b8-4ffe5d89f38f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097408 Jul 20 15:23:05.777293 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 886/2350] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:05.879868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:05.889244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:05.890746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:05.891948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1cf18f62-6870-44c7-8075-c6544cfc9fcf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.154633 Jul 20 15:23:05.892504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1465/2351] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:06.025905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:06.027526 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:06.037922 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.038378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.039181 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.039681 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.070539 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-21dde3ae-372b-4681-b0b1-15a6be5b999b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051294 Jul 20 15:23:06.071100 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 887/2352] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:06.563722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:06.564772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:06.572713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.573161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.573914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.574353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.648231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:06.662926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2920466e-6afd-4bd8-bd4d-4656e23713ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.105304 Jul 20 15:23:06.663534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1466/2353] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:06 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:06.796832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:06.797784 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:06.807370 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.807962 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.808966 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.809535 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.885056 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:06.893393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-753b45ff-1e18-46e3-95a9-ae84fdff16ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.102168 Jul 20 15:23:06.893961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 888/2354] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:06 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:06.909743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:06.910411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:06.917245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.917692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:06.918475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:06.918906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.053594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:07.062215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:07.063755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:07.064930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4e10b28d-283f-4307-b740-8e55f8dbcace tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.159169 Jul 20 15:23:07.065495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1467/2355] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:07.081863 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:07.082814 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:07.089227 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.089669 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.090414 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.090836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.121592 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7d8709fc-d133-4669-b5ac-3fe766f2d05b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043721 Jul 20 15:23:07.122138 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 889/2356] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:07.691888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:07.692826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:07.703593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.704389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.705753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.706525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.770863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:07.779662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e4ed7fec-52e1-4469-8eeb-ce8cac99c6ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.094954 Jul 20 15:23:07.780242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1468/2357] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:07 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:07.914621 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:07.915671 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:07.927125 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.927824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:07.928849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:07.929279 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.000692 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:08.009297 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-bc305987-1bb1-42ce-bbee-7e94a80e560e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.101032 Jul 20 15:23:08.009861 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 890/2358] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:07 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:08.083877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:08.084538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:08.092045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.092547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.093349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.093819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.132651 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:08.133956 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:08.140707 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.141162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.141933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.142373 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.171547 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ebf11d21-6866-46c7-be2b-ee0d76c3c09b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042851 Jul 20 15:23:08.172092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 891/2359] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:08 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:08.227610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:08.236189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:08.237678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:08.238847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bce8bd2e-bfa3-4be8-9b9b-3340e20b8225 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.159190 Jul 20 15:23:08.239717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1469/2360] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:08 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:08.260969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:23:08.261134 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:23:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:23:08.805419 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:08.806602 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:08.816827 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.817581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.818969 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:08.819445 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:08.882140 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:08.885503 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:08.891189 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0ed418e7-4e1b-4543-a79e-acf5e4a0312f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092365 Jul 20 15:23:08.891752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 892/2361] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:08 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:09.026284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:09.026898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:09.033160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.033610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.034360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.034789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.095362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:09.103802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4d2109b3-9530-4c3a-9013-f2886ce4765d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081527 Jul 20 15:23:09.104331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1470/2362] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:09 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:09.187012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:09.188515 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:09.199973 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.200720 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.202012 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.202740 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.234300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f280d6ec-4f2f-4aed-8d94-0c05a167f09a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053972 Jul 20 15:23:09.234835 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 893/2363] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:09.257425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:09.258030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:09.264700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.265140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.265889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.266352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.406271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:09.415061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:09.416560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:09.417712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fc4fe0ff-c643-43f0-a0f8-bba71a30eeb6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.164171 Jul 20 15:23:09.418264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1471/2364] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:09.908749 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:09.909366 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:09.916302 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.916752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.917555 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:09.918010 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:09.982085 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:09.990608 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2f85c199-d821-4af3-9f01-7a96ed510b26 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085999 Jul 20 15:23:09.991197 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 894/2365] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:09 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:10.121959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:10.122562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:10.129695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.130578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.131395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.131837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.201183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:10.213301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d7e4bccb-910d-4bef-8551-daa611ab1919 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094995 Jul 20 15:23:10.214064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1472/2366] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:10.246516 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:10.247394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:10.254430 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.254885 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.255678 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.256122 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.288219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-70762fbe-a524-4030-a743-ba1431552dbb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045781 Jul 20 15:23:10.288761 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 895/2367] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:10.437536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:10.438138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:10.444634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.445077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.445834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:10.446252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:10.609133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:10.618656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:10.620184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:10.621350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1a4f581c-c102-4570-a8e9-66b63eb1c1d0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.188417 Jul 20 15:23:10.621950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1473/2368] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:10 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:11.008309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:11.008946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:11.015305 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.015752 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.016507 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.016934 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.098431 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:11.106882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-77a37e76-57e2-4937-9d07-57c9f00db7e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.102891 Jul 20 15:23:11.107694 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 896/2369] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:11 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:11.237952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:11.238920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:11.250386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.251139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.252421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.253142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.300295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:11.301220 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:11.308146 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.308565 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.309328 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.309758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.317235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:11.328122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5e176d88-993c-4def-9c2f-5c8dd32214b1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.096690 Jul 20 15:23:11.328685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1474/2370] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:11.340745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5a23a55d-56c2-4227-b848-c625b12af833 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044344 Jul 20 15:23:11.341259 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 897/2371] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:11.643541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:11.644504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:11.652128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.652580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.653333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:11.653768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:11.792216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:11.806017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:11.808276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:11.810216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-95dc5045-618e-45f4-a5e0-28792de67f3f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.173259 Jul 20 15:23:11.811114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1475/2372] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:12.128112 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:12.129119 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:12.139452 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.140196 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.141475 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.142205 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.205525 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:12.214109 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fc72a719-49d3-4426-bb3f-052da03c5e79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092507 Jul 20 15:23:12.214663 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 898/2373] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:12 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:12.348436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:12.349475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:12.360764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.361927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.363018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.363794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.364161 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:12.366037 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:12.374343 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.375334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.376124 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.376578 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.409648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-212c6872-64e2-482d-b23f-8f2b365f350d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052285 Jul 20 15:23:12.410361 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 899/2374] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:12.435650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:12.446512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bbbc0ed-ebe6-4bd7-8c8d-1d6450286f3e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.104651 Jul 20 15:23:12.447096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1476/2375] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:12.834933 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:12.835581 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:12.842204 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.842648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.843479 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:12.843911 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:12.991648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:13.000295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:13.001793 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:13.002967 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1f63e7d0-2cd4-4adc-9b95-4d420e7a6c85 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.174442 Jul 20 15:23:13.003563 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 900/2376] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:13.240409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:13.241296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:13.251703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.252455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.253702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.254474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.317158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:13.325198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-44df3c20-2d86-4254-a60b-a17032d10f74 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.091708 Jul 20 15:23:13.325769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1477/2377] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:13 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:13.422092 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:13.423101 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:13.430020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.430476 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.431291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.431725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.462267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-042b5b49-6def-4cbb-805f-2b23ac203402 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044214 Jul 20 15:23:13.462849 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 901/2378] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:13.464061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:13.464704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:13.472050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.472502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.473268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:13.473700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:13.493905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:13.548521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:13.557298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-673b45dd-ac6c-484f-bcd3-9ca78c54922b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097231 Jul 20 15:23:13.557871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1478/2379] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:14.026212 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:14.027261 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:14.037524 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.037980 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.038739 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.039213 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.189982 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:14.199006 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:14.200544 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:14.201730 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-8384ef8b-203f-4ce0-a083-5915f1e72aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.182360 Jul 20 15:23:14.202258 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 902/2380] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:14.347681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:14.348296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:14.354636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.355084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.355906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.356367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.418354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:14.426516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-213072af-6460-4ec9-9264-e923c0c574b7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082879 Jul 20 15:23:14.427071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1479/2381] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:14 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:14.474050 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:14.474968 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:14.486797 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.487309 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.488088 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.488551 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.520226 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-03de5340-fa93-47c1-b87f-7e283d6942d1 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050053 Jul 20 15:23:14.520785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 903/2382] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:14.579196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:14.579918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:14.587752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.588197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.588946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:14.589377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:14.654970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:14.667273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-75ca670b-f41f-4503-8af3-8b7ad99fa8ac tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092475 Jul 20 15:23:14.668041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1480/2383] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:15.221642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:15.222400 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:15.230128 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.230687 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.231676 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.232240 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.370736 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:15.379492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:15.380987 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:15.382152 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9bc00f26-2e36-4a6b-975f-cd4ea7fc3cbb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165496 Jul 20 15:23:15.382675 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 904/2384] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:15.446413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:15.447026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:15.453450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.453873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.454648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.455074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.525689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:15.533454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:15.534390 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:15.534511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7020e027-5830-4a69-b327-0d6f7f742410 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092924 Jul 20 15:23:15.534977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1481/2385] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:15 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:15.542068 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.542511 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.543331 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.543760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.580077 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b9fd0bf5-18dd-4166-b52a-c78036da4be6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050551 Jul 20 15:23:15.580641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 905/2386] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:15 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:15.691827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:15.692834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:15.700296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.700741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.701496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:15.701930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:15.772546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:15.781061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aa06ef5d-98d1-4368-88a8-b24dfc58ff09 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.095705 Jul 20 15:23:15.781626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1482/2387] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:15 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:16.404656 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:16.405706 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:16.414802 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.415292 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.416055 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.416489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.551146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:16.551774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:16.557165 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:16.557982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.558436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.559215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.559650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.565725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:16.567256 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:16.568691 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-0009a7a5-3645-419e-9b6e-c35c9b15ab4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170605 Jul 20 15:23:16.569264 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 906/2388] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:16 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:16.593232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:16.594167 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:16.600492 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.600946 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.601719 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.602160 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.620079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:16.628030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4d74e392-1457-4fd3-b7a0-8f0b0d142b2b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.080828 Jul 20 15:23:16.628636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1483/2389] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:16 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:16.633114 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-7edab906-3159-4ca8-ba1f-43aa6d7a17d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043690 Jul 20 15:23:16.633684 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 907/2390] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:16.800455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:16.801091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:16.809115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.809544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.810587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:16.811019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:16.875798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:16.884390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a6c7d1c0-cc9b-4b41-b682-5b1fc8e05fee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090264 Jul 20 15:23:16.884927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1484/2391] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:17.591735 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:17.592832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:17.603960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.604701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.605953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.606674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.644565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:17.645086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:17.651303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.651748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.652506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.652934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.719105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:17.727591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e606c32-171b-434d-8085-811ee3015e6e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086965 Jul 20 15:23:17.728367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1485/2392] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:17 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:17.732684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:17.733599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:17.740438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.740877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.741633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.742060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.771241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4f5e6703-b491-441b-86c7-807666c2f986 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042015 Jul 20 15:23:17.771777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1486/2393] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:17 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:17.776768 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:17.785826 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:17.787453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:17.788618 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f4fcc907-6596-413e-a342-28654ba11345 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.203436 Jul 20 15:23:17.789169 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 908/2394] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:17.905819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:17.906655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:17.916087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.916569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.917392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:17.917889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:17.982921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:17.991829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c3c8365-a6fc-4107-9bd2-93ef8bb7fab6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092846 Jul 20 15:23:17.992396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1487/2395] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:18.746073 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:18.746682 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:18.753760 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.754233 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.755062 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.755509 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.773319 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:18.783154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:18.784078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:18.790416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.790869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.791697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.792130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.817527 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:18.822359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f988f322-42e4-4684-bead-21a4dc8d053c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043120 Jul 20 15:23:18.823135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1488/2396] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:18.827380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:18.827977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:18.829090 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-f5bd9040-1590-40ff-9209-3eb1253b673f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087204 Jul 20 15:23:18.829683 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 909/2397] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:18 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:18.834071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.834529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.835306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:18.835734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:18.978534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:18.991412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:18.992914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:18.994131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f6dec3d9-599d-4389-b9e2-a6f390498e7f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170216 Jul 20 15:23:18.994739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1489/2398] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:19.009190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:19.009843 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:19.017437 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.017895 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.018666 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.019142 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.080420 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:19.088737 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-df02a99f-2440-4bb2-8db1-da50a0bf0fb0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083464 Jul 20 15:23:19.089334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 910/2399] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:19 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:19.838038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:19.839127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:19.846668 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:19.847421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.847599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:19.847940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.848880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.849392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.854758 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.855300 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.856192 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:19.856701 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:19.881797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bd1644b8-9454-4a65-954d-56037809199e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049527 Jul 20 15:23:19.882372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1490/2400] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:19.922347 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:19.931287 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-776f06b4-3602-4318-8017-3deb12193fd4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088969 Jul 20 15:23:19.931841 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 911/2401] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:19 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:20.016733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:20.017749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:20.028872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.029617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.030874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.031651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.111868 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:20.112387 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:20.119190 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.119627 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.120389 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.120853 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.190572 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:20.194878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:20.198622 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-568a2a20-7b9d-4160-a3e7-c4119b2e84e6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090734 Jul 20 15:23:20.199358 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 912/2402] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:20.203584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:20.205052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:20.206206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0b52903e-169e-423c-9ae9-6bcf828d888c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.196030 Jul 20 15:23:20.206752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1491/2403] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:20.897201 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:20.898599 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:20.908874 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.909571 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.910733 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.911464 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.937507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:23:20.937785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:23:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:23:20.944145 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-1ef620ab-cb7d-4c45-9a03-bcd41f842aa5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053034 Jul 20 15:23:20.944972 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 913/2404] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:20.949451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:20.950129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:20.957100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.957583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:20.958353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:20.958787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.027912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:21.036247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7bfc433f-7269-4a63-8d40-48e1a4d27936 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.092479 Jul 20 15:23:21.036810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1492/2405] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:20 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:21.219548 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:21.220383 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:21.230999 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.231800 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.233063 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.233499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:21.233785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.234566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:21.243392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.243834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.244598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.245061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.335830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:21.344047 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-2215ffee-df10-4347-bdb6-ea2d9728d1b7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.131151 Jul 20 15:23:21.344644 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 914/2406] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:21 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:21.408010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:21.421286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:21.423659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:21.425419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c828309-5fcf-4896-9ba2-b6392fc3eebe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.198338 Jul 20 15:23:21.426159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1493/2407] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:21.959211 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:21.960174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:21.967066 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.967528 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.968291 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:21.968742 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:21.998265 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ced0458e-082d-49e2-94e8-841f6f9b4658 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044705 Jul 20 15:23:21.998818 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 915/2408] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:21 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:22.055110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:22.055755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:22.063169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.063924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.064622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.065177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.129390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:22.137683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-35184952-fde9-49ee-b152-6d258da12066 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087787 Jul 20 15:23:22.138249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1494/2409] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:22 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:22.365830 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:22.367115 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:22.378179 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.378870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.380295 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.381162 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.445061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:22.445668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:22.452220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.452671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.453434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:22.453865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:22.455935 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:22.458000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:22.464771 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-014f7042-4ee9-4c73-9894-4e5d72d17dd7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.106001 Jul 20 15:23:22.465341 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 916/2410] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:22.583180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:22.592445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:22.593959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:22.595103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-afaa2f65-cf5b-40c9-95a0-4ae6063e7983 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.154139 Jul 20 15:23:22.595668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1495/2411] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:22 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:23.014334 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:23.015880 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:23.018839 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.029111 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.029991 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.031454 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.032290 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.064386 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-5ca87555-f50b-4bd0-845b-99283b153529 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056809 Jul 20 15:23:23.064961 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 917/2412] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:23.159609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:23.160653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:23.163772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:23.169502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.169955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.170697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.171183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.236114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:23.248590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b4074e94-1ced-4ca5-9b73-a094e770c72a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.095222 Jul 20 15:23:23.249331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1496/2413] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:23 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:23.485393 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:23.485566 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:23.493971 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.494424 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.495207 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.495834 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.571089 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:23.583886 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e0a0b7a3-e692-4271-aa58-114965ed47c2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.104328 Jul 20 15:23:23.584598 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 918/2414] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:23.620171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:23.620714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:23.627196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.627648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.628383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:23.628836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:23.782701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:23.792995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:23.794456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:23.795649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-add17261-6d54-4b05-b69d-375d1cd908d1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.179468 Jul 20 15:23:23.796209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1497/2415] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:23:24.081920 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:24.083465 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:24.090277 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.090725 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.091520 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.091953 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.121512 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-34681356-61f8-4bd6-ad15-c5e859b31cd7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046259 Jul 20 15:23:24.122070 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 919/2416] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:24.272131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:24.272864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:24.280950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.281481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.282376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.282892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.357107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:24.368339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7666e6f6-d800-4846-b1f1-163f13bec8d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.102600 Jul 20 15:23:24.368903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1498/2417] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:24 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:24.603743 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:24.604369 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:24.612365 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.612891 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.613785 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.614605 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.680232 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:24.688832 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-ea7d1c68-e52d-4a39-8aaf-29fae407f3d7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090383 Jul 20 15:23:24.689405 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 920/2418] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:24.817339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:24.818157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:24.825467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.825959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.826868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:24.827369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:24.975452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:24.984685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:24.986168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:24.987372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f4d3622e-82c1-480f-bc25-8eba2929c407 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.176276 Jul 20 15:23:24.987923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1499/2419] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:25.137496 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:25.139029 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:25.148248 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.148745 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.149502 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.149919 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.179094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-19394bd6-586e-4310-b6c2-386c2747d5ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048481 Jul 20 15:23:25.179648 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 921/2420] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:25.386396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:25.387015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:25.393973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.394426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.395215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.395651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.464207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:25.472400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f95a999b-3e18-4254-924d-1326fb45526f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089907 Jul 20 15:23:25.473564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1500/2421] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:25 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:25.710018 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:25.711094 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:25.722219 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.722870 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.724191 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:25.724917 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:25.793809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:25.802374 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-9b7c5b91-70f8-4b15-b41b-fc5ee1db42f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.099034 Jul 20 15:23:25.803025 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 922/2422] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:26.009295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:26.009960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:26.017618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.018186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.019167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.019711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.157643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:26.166774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:26.168354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:26.169567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-10cb97a2-1b0b-45c2-9be5-8a282edd1e5e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.166027 Jul 20 15:23:26.170263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1501/2423] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:26.192696 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:26.193721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:26.200198 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.200642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.201394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.201821 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.230174 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-fd7c7b53-978a-4ba9-893b-9afd2c7a163a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041417 Jul 20 15:23:26.230721 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 923/2424] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:26.493075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:26.493458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:26.500253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.500705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.501457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.501888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.571693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:26.584303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e89b982f-090a-44b6-abed-0653d32209f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.097007 Jul 20 15:23:26.584837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1502/2425] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:26 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:26.821829 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:26.822440 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:26.829500 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.829947 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.830713 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:26.831144 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:26.899882 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:26.908082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-e1af5f8a-4595-4777-9a56-9818548e5452 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090877 Jul 20 15:23:26.908641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 924/2426] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:27.192387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:27.193347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:27.204142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.204885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.206142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.206862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.244237 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:27.245168 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:27.251705 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.252164 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.252926 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.253356 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.282641 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-c83ab341-9460-4955-a97e-fd84820d91a8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042554 Jul 20 15:23:27.283246 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 925/2427] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:27 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:27.340427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:27.353530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:27.355871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:27.357592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1b94d73d-8d9c-476b-ab12-638e2cfd65b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.171708 Jul 20 15:23:27.358385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1503/2428] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:27.604905 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:27.605965 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:27.616809 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.617556 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.618811 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.619575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.627107 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=100 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:27.687378 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:27.696231 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-b0cb5431-a1ba-4acc-8bfb-90aa91448e88 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.097946 Jul 20 15:23:27.696836 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 926/2429] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:27 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:27.929072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:27.929831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:27.938361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.938914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:27.939905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:27.940446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.013025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:28.021362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-402932f3-94a5-45d0-a821-a1f2f66c0781 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.098640 Jul 20 15:23:28.021933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1504/2430] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:27 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:28.298184 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:28.299722 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:28.310674 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.311489 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.312747 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.313465 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.346662 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO nova.api.openstack.requestlog [None req-78f40581-840e-4003-a830-4640053c42ae tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055146 Jul 20 15:23:28.347208 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: [pid: 124397|app: 0|req: 927/2431] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:28.381160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:28.381807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:28.388098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.388554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.389304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.389739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.533652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:28.542694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:28.544205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:28.545391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-633c1433-6555-483d-a66a-89982604de89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170511 Jul 20 15:23:28.545942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1505/2432] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:28.714575 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.api.openstack.wsgi [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124397) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:28.715095 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.compute.api [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124397) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:28.721203 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.721642 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.722394 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:28.722825 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG oslo_concurrency.lockutils [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124397) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:28.785453 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG nova.policy [None req-6a50886f-a5d8-4c0c-9d07-ca13b4d9aa99 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124397) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:28.791964 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:23:28.794135 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:23:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:23:28.812569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Traceback (most recent call last): Jul 20 15:23:28.812569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers Jul 20 15:23:28.812569 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: timer() Jul 20 15:23:28.812940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ Jul 20 15:23:28.812940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: cb(*args, **kw) Jul 20 15:23:28.812940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: File "/usr/local/lib/python3.8/dist-packages/eventlet/semaphore.py", line 147, in _do_acquire Jul 20 15:23:28.812940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: waiter.switch() Jul 20 15:23:28.812940 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: greenlet.error: cannot switch to a different thread Jul 20 15:23:29.038741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:29.039407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:29.046409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.046858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.047662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.048099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.109508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:29.118085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9f26ee8-333a-45d1-aea3-8f7451fda26d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083377 Jul 20 15:23:29.118676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1506/2433] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:29.360982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:29.362190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:29.369677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.370160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.370920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.371395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.400902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f63e711c-f2c7-4ec3-9a0f-3cf563fd056a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046211 Jul 20 15:23:29.401445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1507/2434] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:29.567030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:29.568119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:29.578439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.579249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.580506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:29.581230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:29.725608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:29.734696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:29.736224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:29.737392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7659b1f4-7f9e-41ef-9519-8d260caf83ac tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.176750 Jul 20 15:23:29.737956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1508/2435] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:29 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:30.139238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:30.140610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:30.148880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.149327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.150083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.150517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.215377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:30.227374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebc77b76-3b6c-4312-a766-b649ffb1daa4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094136 Jul 20 15:23:30.228238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1509/2436] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:30.413844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:30.414677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:30.421380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.421830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.422578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.423004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.454153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bef0c8a9-b321-4d7a-86e0-f02df2f3bcec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044849 Jul 20 15:23:30.454705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1510/2437] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:30.758124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:30.758644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:30.765147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.765597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.766352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:30.766779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:30.908696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:30.913653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:30.921636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:30.923743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:30.925682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e9e73ea8-35f3-451e-bdee-4116be718706 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.172668 Jul 20 15:23:30.926312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1511/2438] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:31.249061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:31.250125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:31.259573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.260020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:31.260769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.261201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:31.321719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:31.330160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-587136d4-e747-47ca-bdae-c3f7ff8fe053 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087652 Jul 20 15:23:31.330712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1512/2439] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:31.469430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:31.470944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:31.478842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.479327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:31.480099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.480535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:31.511617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b61f3a9-3f66-41fb-a096-92d840fc1a6e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048535 Jul 20 15:23:31.512102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1513/2440] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:31 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:31.947279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:31.947988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:31.955570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.956093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:31.956976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:31.957481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:32.102572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:32.115346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:32.117409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:32.119094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a2a306c1-6b28-4fa2-9b2d-6e76394983ed tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.176676 Jul 20 15:23:32.119852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1514/2441] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:32.351181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:32.352270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:32.362831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:32.363633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:32.365289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:32.365750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:32.429943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:32.442371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40b996ca-34ad-4226-901c-926aa695b5bc tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.097378 Jul 20 15:23:32.443125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1515/2442] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:32.523351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:32.524275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:32.531113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:32.531560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:32.532317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:32.532745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:32.562419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f45ace43-d32e-4cd6-bbc1-fb414a0832d0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043080 Jul 20 15:23:32.562964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1516/2443] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:32 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:33.140826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:33.141673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:33.149539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.149985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.150733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.151192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.295534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:33.304521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:33.306012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:33.307216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f5fc8b62-3e3a-4153-bc4a-322b7c0ac0fb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.172759 Jul 20 15:23:33.307777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1517/2444] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:33.468286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:33.469368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:33.478318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.478819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.479619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.483506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.543177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:33.551670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-def9e851-53d8-439a-8758-a0fcdbc5d756 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089882 Jul 20 15:23:33.552245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1518/2445] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:33.573661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:33.574700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:33.580795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.581244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.581996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:33.582426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:33.610805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ff96a2f6-bff3-4313-97d0-30f89a89dbce tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040802 Jul 20 15:23:33.611371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1519/2446] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:34.330049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:34.331155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:34.342278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.343027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.344321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.345130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.497142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:34.507079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:34.508717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:34.509919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9fcf4e00-d7e3-4a9c-b2d5-539109d5b3db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.186878 Jul 20 15:23:34.510501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1520/2447] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:23:34.568604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:34.569197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:34.575765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.576236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.576983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.577411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.639677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:34.648375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d4addd51-2914-475a-aeda-d3841735e4c8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083523 Jul 20 15:23:34.648949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1521/2448] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:34 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:34.653415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:34.654316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:34.660318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.660764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.661520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:34.661954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:34.692668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbd89fe2-c639-4c93-a97d-b51f6b75dc83 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042702 Jul 20 15:23:34.693262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1522/2449] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:34 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:35.530181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:35.530805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:35.538086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.538537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.539334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.539767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.688221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:35.698712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:35.700236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:35.701397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e7ada13d-2956-4855-a3da-231e1e87764c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.175239 Jul 20 15:23:35.701965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1523/2450] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:35 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:35.706349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:35.706937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:35.713250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.713697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.714446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.714875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.802638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:35.815019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3425d79f-9687-43ac-965a-a36d5c3d8295 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.111809 Jul 20 15:23:35.815935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1524/2451] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:35.821249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:35.822162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:35.828263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.828711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.829466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:35.829903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:35.860227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da89c5ba-aceb-457b-ae4f-341e7395f634 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043064 Jul 20 15:23:35.860787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1525/2452] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:35 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:36.719502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:36.720239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:36.727666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.728123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:36.728880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.729312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:36.874343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:36.883438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:36.884894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:36.886052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-37196187-b075-45d9-a855-0a1c444fdc4a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170743 Jul 20 15:23:36.886583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1526/2453] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:36.890864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:36.891480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:36.897527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.898159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:36.898917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.899379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:36.971370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:36.979679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-89d24a11-d0a1-4e6f-829a-68d9124fbee7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092105 Jul 20 15:23:36.980207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1527/2454] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:36.984656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:36.985685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:36.992127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.992567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:36.993310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:36.993732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:37.010211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:37.023440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-856e1c69-43a1-47d9-b4b8-9bf295259903 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042086 Jul 20 15:23:37.024022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1528/2455] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:37.908248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:37.909328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:37.920215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:37.920955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:37.922196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:37.922911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:38.063774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:38.072860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:38.074339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:38.075534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57c1002a-3330-4570-80c0-12db13273217 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.173913 Jul 20 15:23:38.076091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1529/2456] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:38.080407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:38.081099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:38.087158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:38.087606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:38.088367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:38.088800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:38.160021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:38.168170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a467249c-c40a-41b3-a2e2-81efe20182be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.091122 Jul 20 15:23:38.168728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1530/2457] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:38.173207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:38.174129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:38.179978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:38.180415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:38.181165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:38.181547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:38.209635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27edbee5-d69c-4c77-ad5b-fa8b71642fa3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039935 Jul 20 15:23:38.210179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1531/2458] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:38.719155 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_api:SELECT=23 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:38.790472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: DEBUG dbcounter [-] [124397] Writing DB stats nova_cell1:SELECT=16 {{(pid=124397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:39.096926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:39.097751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:39.107583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.108213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.109287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.109899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.244905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:39.253946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:39.255496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:39.256667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ef5f8364-93b3-4734-ab2e-f7ea75ad4e7c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165346 Jul 20 15:23:39.257223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1532/2459] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:39.261499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:39.262092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:39.267804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.268252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.268999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.269455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.333013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:39.341714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a19582e2-65f4-4ced-9836-ff6e5ba834be tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083519 Jul 20 15:23:39.342276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1533/2460] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:39.347141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:39.348058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:39.354144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.354571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.355376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:39.355815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:39.389947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53bc37a2-0fd1-47a1-91fc-7395c117ac21 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046454 Jul 20 15:23:39.390653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1534/2461] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:40.279195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:40.280228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:40.289517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.289926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.290719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.291184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.424540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:40.437591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:40.439934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:40.441563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e450cb5e-111a-4a88-919b-0588bfc72cca tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168884 Jul 20 15:23:40.442301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1535/2462] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:40.446725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:40.447353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:40.453002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.453479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.454252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.454686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.524124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:40.532545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0018bcbb-c33c-4273-a6ee-b624aee25642 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089234 Jul 20 15:23:40.533055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1536/2463] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:40 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:40.537619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:40.538527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:40.544332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.544785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.545534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:40.545971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:40.576638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-20f8bf8f-2261-43a3-bb10-f2e1945c5e90 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042560 Jul 20 15:23:40.577214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1537/2464] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:40 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:41.462558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:41.463558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:41.473275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.473904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.474951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.475675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.608199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:41.617312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:41.618795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:41.620008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1d899dc-092c-43eb-b67b-6add39b5e79d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.163352 Jul 20 15:23:41.620567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1538/2465] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:41 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:41.624976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:41.625509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:41.631314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.631779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.632530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.632962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.706255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:41.715038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f077cd96-e4ca-4756-9699-cc86759118fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093529 Jul 20 15:23:41.715639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1539/2466] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:41.720130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:41.721038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:41.727016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.727494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.728240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:41.728673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:41.758521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1608754-5718-47d8-a945-97e8ef9748e4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041877 Jul 20 15:23:41.759081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1540/2467] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:42.637737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:42.638436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:42.645464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.645920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.646680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.647150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.783650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:42.792792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:42.794316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:42.795542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-276e8a7a-cf0e-476b-8085-8685a89a1875 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161786 Jul 20 15:23:42.796128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1541/2468] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:42.800744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:42.801392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:42.807519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.807972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.808732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.809167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.870809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:42.879373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b29098bf-cef0-456b-bbb9-037d56691343 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082215 Jul 20 15:23:42.879983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1542/2469] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:42.884553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:42.885459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:42.893838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.894423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.895650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:42.896152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:42.927006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4fe6c90b-3980-42dc-9603-e612569768d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046000 Jul 20 15:23:42.927611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1543/2470] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:42 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:43.818069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:43.819185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:43.830679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:43.831537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:43.832853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:43.833692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:43.979972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:43.988796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:43.990338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:43.991519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24607a0a-f631-4c83-9959-05a60faec098 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.180367 Jul 20 15:23:43.992072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1544/2471] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:23:43.996315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:43.996906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:44.002713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:44.003216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:44.004056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:44.004488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:44.008567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.066832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:44.075082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fca7feef-aa6b-4818-b8d3-441d9cf3f2d8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081956 Jul 20 15:23:44.075594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1545/2472] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:44.080149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:44.081055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:44.082616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:44.088021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:44.088465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:44.089209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:44.089639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:44.119021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-27bce911-ad10-4860-8314-5e6136b9471f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042433 Jul 20 15:23:44.119603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1546/2473] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:45.012618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:45.013464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:45.023795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.024459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.025539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.026210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.161166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:45.170067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:45.171596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:45.172776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b72fe131-44a4-46cb-b79b-cba9e55343c2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.166110 Jul 20 15:23:45.173386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1547/2474] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:23:45.177728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:45.178362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:45.185006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.185551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.186716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.187663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.255250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:45.263571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-254bc9c5-3264-4f3e-a219-a09f33db061d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089241 Jul 20 15:23:45.264138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1548/2475] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:45.268665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:45.269575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:45.275595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.276060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.276810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:45.277239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:45.306391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-006e1304-a906-4db2-82e6-74dbde32a0ff tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041262 Jul 20 15:23:45.306937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1549/2476] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:46.192146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:46.192915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:46.199912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.200355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.201119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.201553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.333214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:46.342216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:46.343862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:46.345088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f67b3fa-f4e9-4de6-a436-9a69b7a3aa8d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.157056 Jul 20 15:23:46.345650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1550/2477] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:46.350108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:46.350751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:46.357161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.357608 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.358385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.358821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.422963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:46.436123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aaea8fbd-e2ec-4498-966a-120bb96d13d5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089502 Jul 20 15:23:46.436707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1551/2478] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:46.441683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:46.442687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:46.450336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.450785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.451581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:46.452011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:46.495551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5963c39e-0954-4c49-8eeb-241edcb06b93 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057406 Jul 20 15:23:46.496399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1552/2479] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:46 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:47.367264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:47.368118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:47.376575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.377456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.378037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.378612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.520146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:47.529015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:47.530513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:47.531713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eea77548-9c20-4b4f-ad83-a8fee96f0919 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170735 Jul 20 15:23:47.532251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1553/2480] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:47.536556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:47.537185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:47.543207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.543646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.544401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.544840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.605066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:47.613209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-321a24bb-1444-47d0-8de1-62dc3814fb59 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.079981 Jul 20 15:23:47.614091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1554/2481] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:47 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:47.618710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:47.619591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:47.625415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.625864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.626709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:47.627169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:47.656561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36dab026-d9bd-49ae-878b-e7c94de8480b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041365 Jul 20 15:23:47.657120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1555/2482] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:48.550992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:48.551709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:48.559070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.559550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.560330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.560786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.689987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:48.699251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:48.700837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:48.702205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6567a3b8-bfb6-452e-8f2f-c8cb1ee43b68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.155155 Jul 20 15:23:48.702749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1556/2483] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:48 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:48.707492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:48.708100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:48.714043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.714499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.715291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.715728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.779014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:48.787536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2263db7c-be83-4aaf-aae2-ccad8c715f1b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083837 Jul 20 15:23:48.788189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1557/2484] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:48.797609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:48.798538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:48.806101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.806832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.808081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:48.808795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:48.847767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9a08c63-8121-4427-a4f4-7ef5b40dfc1e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058447 Jul 20 15:23:48.848341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1558/2485] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:49.725062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:49.726377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:49.737360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.738101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:49.739820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.740252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:49.873847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:49.882603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:49.884128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:49.885290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b153a314-eccf-49b9-8603-13c372d74270 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.166644 Jul 20 15:23:49.885835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1559/2486] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:49.890108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:49.890714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:49.896620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.897067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:49.897820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.898253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:49.966037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:49.976074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c38798f1-4a41-41b8-9a38-5a42c55d5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089094 Jul 20 15:23:49.976553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1560/2487] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:49.980984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:49.981923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:49.989169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.989767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:49.990910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:49.991597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:50.021026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:50.025779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-094b5eaf-e05d-4f99-b736-f1facf3e75ac tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048246 Jul 20 15:23:50.026326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1561/2488] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:50.908584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:50.909757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:50.917470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:50.917920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:50.918668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:50.919107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:51.061511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:51.071512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:51.072985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:51.074199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5d38786e-63a0-4580-8343-f97ff8f3a281 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.172743 Jul 20 15:23:51.074758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1562/2489] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:51.079082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:51.079713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:51.086099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:51.086710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:51.087982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:51.088580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:51.166742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:51.179042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9e77c207-9a7b-4f3d-8ed3-163aef65af61 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.103225 Jul 20 15:23:51.179695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1563/2490] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:51.184371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:51.185279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:51.191331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:51.191784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:51.192544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:51.193025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:51.222957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-91371610-3071-45ec-97d9-99ba555579d9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042230 Jul 20 15:23:51.223556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1564/2491] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:52.093232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:52.093896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:52.100766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.101238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.101995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.102428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.237424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:52.250969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:52.253228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:52.254783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7305d290-44ec-47e6-8a94-217a7ba78711 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165847 Jul 20 15:23:52.255591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1565/2492] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:52.259969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:52.260562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:52.266311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.266760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.267551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.267985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.335568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:52.348180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92f410cc-071f-4b5e-9861-bddb392b6de2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.091238 Jul 20 15:23:52.349012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1566/2493] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:52.353472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:52.354414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:52.360349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.360793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.361545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:52.361975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:52.396045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-137655a7-8cfc-43dc-9865-aa4a4e0fe807 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045977 Jul 20 15:23:52.396589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1567/2494] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:53.275083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:53.275801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:53.283084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.283605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.284362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.284798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.425002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:53.433999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:53.435499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:53.436660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ecec6e6d-6c26-464d-8f41-a76de67e4a16 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165613 Jul 20 15:23:53.437193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1568/2495] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:53.441500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:53.442085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:53.447956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.448403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.449179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.449613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.512303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:53.520358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9b16a265-d247-422c-bd6e-b639274f746e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082242 Jul 20 15:23:53.520963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1569/2496] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:53 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:53.525650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:53.526495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:53.532204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.532642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.533408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:53.533831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:53.563447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aeafdd89-cb7f-4128-b85e-e4f869b6dc99 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041464 Jul 20 15:23:53.563981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1570/2497] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:53 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:54.458973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:54.460108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:54.471035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.471834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.473177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.473985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.617109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:54.626953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:54.628461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:54.629615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0a45b2e7-e00a-4c37-9c1e-e358474518cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.177157 Jul 20 15:23:54.630284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1571/2498] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:54 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:54.634643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:54.635279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:54.642410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.643212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.644499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.645234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.715356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:54.723914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bfa72b3-c50f-4239-b892-80f7aa6fffda tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092596 Jul 20 15:23:54.724473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1572/2499] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:54 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:54.729122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:54.730045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:54.735806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.736245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.736987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:54.737413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:54.766881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e46c0bc2-f297-44e8-aa61-700c44815329 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041435 Jul 20 15:23:54.767440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1573/2500] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:54 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:55.652286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:55.653249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:55.664260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.665008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.666261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.666985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.810799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:55.820055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:55.821554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:55.822702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ddb18375-c2ea-4d1e-83f9-1eedeacaad54 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.177262 Jul 20 15:23:55.823284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1574/2501] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:55 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:55.827544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:55.828140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:55.833889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.834331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.835198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.835651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.906588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:55.914873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-652a6312-fd4b-42ea-af88-c1cae6a096d2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.090662 Jul 20 15:23:55.915472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1575/2502] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:55 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:55.919864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:55.920763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:55.926495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.927079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.927832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:55.928258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:55.957228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-290b2e4d-124f-41e7-a6d1-6236dce1d49d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040781 Jul 20 15:23:55.957754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1576/2503] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:55 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:56.843431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:56.844414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:56.852158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:56.852571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:56.853312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:56.853735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:56.990644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:57.003850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:57.005801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:57.006957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c205841-b98f-4893-a362-515ee73b78b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.168839 Jul 20 15:23:57.007543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1577/2504] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:57.011808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:57.012439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:57.018130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:57.018580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:57.019369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:57.019803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:57.039141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:23:57.087957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:57.096487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ed33b4aa-ab94-415c-943b-f76de891c803 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087932 Jul 20 15:23:57.096990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1578/2505] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:57 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:57.101437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:57.102352 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:57.108264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:57.108712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:57.109458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:57.109890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:57.139346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fbc711c5-59d0-4167-a461-23346bdfd18d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041333 Jul 20 15:23:57.139893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1579/2506] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:57 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:58.029496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:58.030668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:58.041847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.042582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.043876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.044587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.188865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:58.197591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:58.199076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:58.200261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-766c8f2e-363c-402f-a851-dbaeb945dfde tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.177571 Jul 20 15:23:58.200819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1580/2507] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:58 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:58.205218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:58.205824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:58.211734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.212174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.212917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.213347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.277837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:58.286299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be4220c8-3302-4049-9efd-1cb29e6693d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084376 Jul 20 15:23:58.286759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1581/2508] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:58 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:58.291313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:58.292206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:58.298065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.298513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.299289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:58.299748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:58.329939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-28068fc5-33cd-480c-8ddd-3a2e1566e225 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042193 Jul 20 15:23:58.330481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1582/2509] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:58 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:23:59.222323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:59.223686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:59.230554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.230992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.231759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.232186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.375703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:59.386663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:59.388472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:59.390170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6a850480-523c-4898-91f0-dab56fd2a3ad tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.174071 Jul 20 15:23:59.390888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1583/2510] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:23:59 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:23:59.395330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:59.396000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:59.402198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.402639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.403424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.403848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.473535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:23:59.481909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbdb012e-e90b-4ad3-b794-aaa00c739ac5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089866 Jul 20 15:23:59.482302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1584/2511] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:23:59 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:23:59.486973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:23:59.487884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:23:59.494254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.494695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.495474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:23:59.495899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:23:59.525177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bd1ca26-ce39-4add-b2bf-b786ed885022 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041893 Jul 20 15:23:59.525748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1585/2512] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:23:59 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:00.412906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:00.414078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:00.424566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.425322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.426562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.427233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.576190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:00.589810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:00.592212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:00.593937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f9fcfea0-0855-4afc-87fc-db3f4bb58c2b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.187567 Jul 20 15:24:00.594698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1586/2513] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:00.600440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:00.601044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:00.606940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.607437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.608194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.608626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.676826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:00.684985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-70861fc0-2b66-4b8b-a580-e922700d3084 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089091 Jul 20 15:24:00.685547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1587/2514] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:00 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:00.690006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:00.690944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:00.696965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.697416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.698170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:00.698599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:00.727473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-33acf19e-9b97-4c25-ba0f-ca13298b7ba8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040905 Jul 20 15:24:00.728023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1588/2515] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:01.618158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:01.619282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:01.630216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.630958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.632274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.632991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.779651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:01.792720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:01.795230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:01.797139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eb3a2b78-bdc7-433a-b834-20b0b7bb5efe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.185750 Jul 20 15:24:01.798008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1589/2516] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:01 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:01.803850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:01.804445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:01.810290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.810742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.811533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.811966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.884409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:01.892977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c298f8e-27a9-48b5-9cac-1313717e47ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093797 Jul 20 15:24:01.893518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1590/2517] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:01 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:01.898203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:01.900103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:01.905182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.905624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.906382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:01.906818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:01.938282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ffc18cab-b392-4c92-afe7-bd1f351e8c7b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043700 Jul 20 15:24:01.938845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1591/2518] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:02.820241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:02.821159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:02.828619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:02.829121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:02.829959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:02.830448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:02.964674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:02.973350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:02.974880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:02.976073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c5784dc-a828-44ac-8932-aeb42c24dbab tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.161247 Jul 20 15:24:02.976634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1592/2519] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:02 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:02.980976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:02.981591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:02.987420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:02.988937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:02.989860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:02.990325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:03.051698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:03.060688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d5cbee6e-0c7d-44f9-90b3-fa8a7dca673e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083100 Jul 20 15:24:03.061285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1593/2520] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:02 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:03.066000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:03.066914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:03.073018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:03.073404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:03.074161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:03.074604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:03.105232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1630490e-6d88-49b2-9ec2-c468751c7b91 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042923 Jul 20 15:24:03.105796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1594/2521] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:03.999021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:04.000122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:04.011127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.011858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.013080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.013832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.020342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.159797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:04.168603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:04.170118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:04.171338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6369a13f-0531-4ac7-87a1-73212a1e32c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.178755 Jul 20 15:24:04.171892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1595/2522] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:03 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:04.176215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:04.176896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:04.178520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:04.183946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.184388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.185134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.185592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.247789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:04.256345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-258f2619-083c-4a63-8b4b-d9613ab1c029 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083443 Jul 20 15:24:04.256854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1596/2523] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:04 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:04.261439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:04.262328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:04.268671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.269120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.269879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:04.270313 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:04.300457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d2fcf334-2626-4bc9-af0a-60a76ad7dbea tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042604 Jul 20 15:24:04.301007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1597/2524] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:04 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:05.190117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:05.190740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:05.197770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.198215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.198979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.199513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.342113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:05.355492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:05.357758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:05.359624 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a49b0793-297e-4935-85f9-b5d43c186e68 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.173302 Jul 20 15:24:05.360488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1598/2525] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:05 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:05.365194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:05.365794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:05.371550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.371999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.372757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.373190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.436713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:05.445001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-10d7d846-29a5-4552-98d0-50f9d1544b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083505 Jul 20 15:24:05.445490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1599/2526] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:05 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:05.449936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:05.450843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:05.456517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.456964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.457749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:05.458208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:05.490550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bdaa5744-100f-4f98-9b74-355510f979af tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044033 Jul 20 15:24:05.491368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1600/2527] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:06.383127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:06.384307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:06.395478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.395930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.396684 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.397113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.532789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:06.541741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:06.543267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:06.544444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e03ada8d-82bd-4b48-a946-608b6a22c178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167817 Jul 20 15:24:06.544997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1601/2528] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:06 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:06.549252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:06.549842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:06.555834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.556296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.557022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.557450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.625647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:06.634448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7bad7193-75c0-423c-858e-9838997a1863 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.088448 Jul 20 15:24:06.634945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1602/2529] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:06 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:06.639392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:06.640299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:06.646207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.646651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.647535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:06.647964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:06.679409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d38c9d6e-131e-4ab5-ba38-f79250c06c66 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043392 Jul 20 15:24:06.679974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1603/2530] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:07.567305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:07.567930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:07.575250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.575729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.576519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.576997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.722230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:07.734269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:07.736611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:07.738671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d8b1238f-599c-4509-8755-dc854d8ab424 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.177688 Jul 20 15:24:07.739496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1604/2531] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:07 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:07.743776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:07.744374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:07.750498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.750945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.751732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.752167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.814279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:07.822909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3aa6db7b-dc8d-43d7-81d8-a6d63c3c5d00 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082379 Jul 20 15:24:07.832490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1605/2532] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:07 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:07.832490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:07.832490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:07.834549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.834990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.835772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:07.836200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:07.864556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0cbb937-c0b8-4ce3-83ef-e15fce2873d5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040183 Jul 20 15:24:07.865092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1606/2533] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:08.757668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:08.758308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:08.765210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:08.765657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:08.766411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:08.766841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:08.906922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:08.915904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:08.917399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:08.918571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-473cbcb5-1c72-4a6e-a5ab-8343e7ef5f1b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.165056 Jul 20 15:24:08.919162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1607/2534] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:08 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:08.923662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:08.924268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:08.930101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:08.930552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:08.931337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:08.931774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:08.996644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:09.005796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eb405706-200d-459a-84c9-df5490e8e9a7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085628 Jul 20 15:24:09.006462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1608/2535] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:08 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:09.011148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:09.012035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:09.017973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:09.018419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:09.019200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:09.019634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:09.048242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-17ee5260-fe65-4f4c-a883-87de12bb859f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040774 Jul 20 15:24:09.048774 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1609/2536] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:09 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:09.937857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:09.938489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:09.945285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:09.945727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:09.946520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:09.946943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:10.077605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:10.086858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:10.088391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:10.089590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-221190b9-696e-44e2-832e-22605509ea4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.155644 Jul 20 15:24:10.090152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1610/2537] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:09 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:10.094439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:10.095039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:10.101057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:10.101515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:10.102272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:10.102708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:10.168770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:10.172464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:10.178287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bc01f3b1-736b-4daa-ad4a-686cb90e2f06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087090 Jul 20 15:24:10.179168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1611/2538] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:10.183620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:10.184557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:10.191351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:10.191801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:10.192633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:10.193072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:10.224703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-81e4cf9d-c850-45a4-85f1-c6594d3e9e2e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044720 Jul 20 15:24:10.225246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1612/2539] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:10 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:11.108105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:11.108839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:11.116015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.116460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.117198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.117625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.316435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:11.325764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:11.327266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:11.328419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-13046644-1463-47df-b4a7-fd290754ab5f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.224245 Jul 20 15:24:11.328965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1613/2540] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:11 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:11.333260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:11.333853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:11.339581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.340024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.340765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.341192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.408946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:11.417125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92b5293e-d5da-4060-a647-38d66211b7c1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.087201 Jul 20 15:24:11.417688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1614/2541] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:11 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:11.422107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:11.423015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:11.429281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.429721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.430464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:11.430912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:11.459514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bbd3b174-d4b7-4f8b-80ee-87d693f582b4 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040798 Jul 20 15:24:11.460073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1615/2542] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:11 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:12.350773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:12.351926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:12.363085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.363974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.364730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.365156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.500233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:12.509102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:12.510588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:12.511785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d79575b6-2935-46ba-a822-dcdc9c177905 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167487 Jul 20 15:24:12.512346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1616/2543] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:12 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:12.516637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:12.517232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:12.523173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.523643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.524394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.524829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.585678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:12.594114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a02605c7-f0a7-4788-bfb4-3a4417e2b2ad tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080642 Jul 20 15:24:12.594730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1617/2544] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:12 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:12.600020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:12.601127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:12.606916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.607391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.608135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:12.608563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:12.638933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84bc9dcc-3f8b-4c04-ae65-3e3bc45fea38 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043119 Jul 20 15:24:12.639513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1618/2545] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:12 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:13.530387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:13.530971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:13.538054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.538484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.539281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.539735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.673688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:13.683392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:13.684875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:13.686035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a171e171-8ca2-4d66-879b-122cfe13e7d9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.159654 Jul 20 15:24:13.686594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1619/2546] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:13 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:13.690899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:13.691525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:13.699471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.700098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.701249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.701948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.768593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:13.777301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5d9c3b5d-afa8-477a-92d5-08d593bbac5d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.089705 Jul 20 15:24:13.777875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1620/2547] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:13 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:13.782447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:13.783410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:13.789126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.789573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.790312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:13.790745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:13.820252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-08e26a5f-e740-4229-af30-b4efde4536aa tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041313 Jul 20 15:24:13.820790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1621/2548] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:13 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:14.707473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:14.708255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:14.716584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.717101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.717976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.718477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.856415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:14.870216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:14.872504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:14.874233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ca3b440-d469-423f-b4bc-7e12d2aad38f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.171112 Jul 20 15:24:14.874996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1622/2549] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:14 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:14.879447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:14.880181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:14.886175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.886617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.887388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.887788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.948689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:14.956585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9691a91b-34f6-4b55-8039-1ca334217e0e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080604 Jul 20 15:24:14.957163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1623/2550] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:14 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:14.961872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:14.962776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:14.968314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.968750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.969488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:14.969912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:14.998981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6eaacd39-8784-408c-98d3-259cc7067108 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040727 Jul 20 15:24:14.999553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1624/2551] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:14 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:15.893075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:15.893784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:15.900911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:15.901358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:15.902139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:15.902526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:16.052555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:16.065936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:16.068385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:16.070158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-be68bd6e-069f-447d-ba61-2669d6b4e551 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.180750 Jul 20 15:24:16.070934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1625/2552] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:15 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:16.075267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:16.075871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:16.082426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:16.082970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:16.083789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:16.084203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:16.156214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:16.169051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-19ee3b3e-4c1d-43b8-a32d-5712807033b8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.096753 Jul 20 15:24:16.169963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1626/2553] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:16 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:16.176132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:16.177187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:16.183586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:16.184036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:16.184776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:16.185232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:16.215426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-732ede97-bb5d-4233-9953-f4a0279bbea7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044197 Jul 20 15:24:16.216204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1627/2554] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:16 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:17.092901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:17.093980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:17.102668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.103145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.103947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.104427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.123191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:17.243470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:17.252447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:17.253961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:17.255364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b240a57a-9b47-47e6-918b-aab96eacf294 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.169203 Jul 20 15:24:17.255906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1628/2555] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:17 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:17.260166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:17.260758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:17.266510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.266953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.267725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.268155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.340824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:17.354156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-107dda57-0fae-46e5-82e3-8b8b61cff7d3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.096903 Jul 20 15:24:17.354959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1629/2556] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:17 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:17.362224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:17.364042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:17.370121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.370568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.371366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:17.371801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:17.400947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8ad7565f-44fe-406c-be79-8ea441026f2a tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044651 Jul 20 15:24:17.401499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1630/2557] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:17 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:18.277987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:18.279105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:18.290330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.291310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.292775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.293509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.437537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:18.446139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:18.447694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:18.448852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b438e183-7b02-4554-a42f-33ded85a2479 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.177543 Jul 20 15:24:18.449380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1631/2558] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:18 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:18.453641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:18.454239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:18.459905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.460347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.461095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.461523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.523695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:18.532197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76e0826e-211d-4745-a5c7-8ec570d288d0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081788 Jul 20 15:24:18.532692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1632/2559] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:18 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:18.537193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:18.538108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:18.543896 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.544338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.545084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:18.545514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:18.574116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d855195d-a7b4-4c8a-b029-f844212f47ef tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040421 Jul 20 15:24:18.574662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1633/2560] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:18 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:19.470987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:19.472116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:19.480007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.480475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.481284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.481735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.622494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:19.631672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:19.633165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:19.634330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-77b8c39c-8b9d-4b4c-a4a8-5f7aa52d6f9b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.169852 Jul 20 15:24:19.634895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1634/2561] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:19 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:19.639242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:19.639886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:19.646612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.647109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.648226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.648803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.725709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:19.734201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3e0f6a64-ec6b-4c7a-9941-5ba5f212280e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.098341 Jul 20 15:24:19.734764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1635/2562] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:19 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:19.739606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:19.740483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:19.746127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.746572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.747347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:19.747780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:19.776547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-13ba2a29-663a-440a-b7ad-db3540f21356 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040777 Jul 20 15:24:19.777094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1636/2563] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:19 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:20.657226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:20.658288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:20.669308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.670009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.671315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.672095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.822018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:20.830627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:20.832141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:20.833345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a5309457-468f-4c7c-8329-dc117819fba5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.182744 Jul 20 15:24:20.833901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1637/2564] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:20 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:20.838243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:20.838839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:20.844575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.845022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.845768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.846196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.909558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:20.917621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5ed3dafe-5a9d-456f-9460-23a9ab0c95ee tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082783 Jul 20 15:24:20.918156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1638/2565] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:20 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:20.922606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:20.923539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:20.929566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.930013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.930767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:20.931247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:20.959723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c5effea-ba3a-4e96-a619-a74809984ddf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040580 Jul 20 15:24:20.960269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1639/2566] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:20 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:21.851724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:21.852299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:21.859175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:21.859628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:21.860368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:21.860801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:21.994017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:22.002664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:22.004230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:22.005388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8cd88102-668f-4ad1-8be5-bb1a087d7d06 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.157673 Jul 20 15:24:22.005947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1640/2567] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:21 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:22.010221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:22.010821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:22.016513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:22.016961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:22.017709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:22.018139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:22.080110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:22.088308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9cf04f78-608a-4e7b-bc3c-dc0cfc1f1fbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.081424 Jul 20 15:24:22.088872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1641/2568] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:22 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:22.093425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:22.094248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:22.100117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:22.100559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:22.101314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:22.101744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:22.130827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-182d98a3-0787-4c79-8a7c-5f99c463e02b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040947 Jul 20 15:24:22.131412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1642/2569] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:22 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:23.026213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:23.027813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:23.037048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.037484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.038240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.038664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.190954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:23.199874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:23.201349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:23.202513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-55dfeb8b-211f-49a0-8586-f8861dc3a858 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.181615 Jul 20 15:24:23.203090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1643/2570] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:23 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:23.207400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:23.208000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:23.213759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.214210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.214958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.215617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.275944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:23.283931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-39e338df-2ea3-4bb9-a54e-9494bdaf1147 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.079816 Jul 20 15:24:23.284425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1644/2571] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:23 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:23.288907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:23.289788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:23.295565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.296007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.296761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:23.297162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:23.300736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:23.327846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-014dca51-4dfa-4ccb-828a-d0ea5d0dddec tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042381 Jul 20 15:24:23.328535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1645/2572] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:23 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:24.224609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:24.225764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:24.229256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:24.238443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.239078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.239867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.240303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.376888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:24.389407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:24.391703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:24.393370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-755c5c8f-a5eb-4e00-a51e-6070dbd7929f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.175154 Jul 20 15:24:24.394126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1646/2573] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:24 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:24.398647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:24.399304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:24.405111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.405563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.406325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.406759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.472649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:24.481443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-173e156e-eab2-4b4a-b555-a6e50272cd06 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086174 Jul 20 15:24:24.481912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1647/2574] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:24 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:24.486507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:24.487766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:24.493775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.494223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.495020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:24.495553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:24.525554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8347623f-7aa0-486e-81f2-e90f1b110031 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042611 Jul 20 15:24:24.526089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1648/2575] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:24 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:25.414189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:25.414768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:25.421971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.422464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.423350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.423812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.561083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:25.570656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:25.572281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:25.573476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-015229d0-3264-4d8b-ad03-909a25304bbf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.163793 Jul 20 15:24:25.574151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1649/2576] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:25 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:25.578682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:25.579353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:25.585797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.586243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.587007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.587473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.650743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:25.658868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e6a7ea2-5286-47fc-9146-b58e9306bafd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083731 Jul 20 15:24:25.659470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1650/2577] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:25 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:25.663926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:25.664894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:25.670816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.671327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.672099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:25.672565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:25.702455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-68c4ffed-cb5d-4854-88dd-0fc049bf761b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041990 Jul 20 15:24:25.703013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1651/2578] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:25 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:26.596544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:26.597597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:26.608794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.609532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.610777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.611540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.759963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:26.768422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:26.769883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:26.771069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72429580-4848-4cf9-86d8-fae8445338f5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.181125 Jul 20 15:24:26.771704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1652/2579] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:26 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:26.776006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:26.776611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:26.782207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.782655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.783422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.783848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.846564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:26.855094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ce9289b-a851-4819-9249-ce4c2c63ec39 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082443 Jul 20 15:24:26.855676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1653/2580] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:26 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:26.860171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:26.861081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:26.867151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.867577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.868338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:26.868768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:26.898296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bbf46630-5ede-4e4e-bd6a-1647c9fbe31f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041628 Jul 20 15:24:26.898849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1654/2581] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:26 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:27.791015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:27.791910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:27.799111 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:27.799613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:27.800412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:27.800879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:27.935395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:27.944976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:27.946457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:27.947653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72aae987-aa48-4031-94d7-5fa309b10d95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.160979 Jul 20 15:24:27.948243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1655/2582] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:27 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:27.952568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:27.953167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:27.959278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:27.959731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:27.960482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:27.960916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.035350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:28.043810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fd623d28-8846-4324-b351-89777646783e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.094630 Jul 20 15:24:28.044369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1656/2583] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:27 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:28.048890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:28.049796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:28.055603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.056047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.056796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.057221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.089279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5f4efaf8-d1e7-491c-aa4b-9bb9a6caa448 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043545 Jul 20 15:24:28.090012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1657/2584] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:28 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:28.852515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:28.853509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:28.863162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.863890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.865106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.865750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.940628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:28.948879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-02d18e83-5545-41de-83d0-afaeb0c7f074 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.176687 Jul 20 15:24:28.949446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1658/2585] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:28 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:28.965506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:28.966102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:28.972182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.972627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:28.973573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:28.974003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.104411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:29.115919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:29.118372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:29.120060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45cf7df7-d48c-4a05-b7e7-40fa8206b8ba tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.157769 Jul 20 15:24:29.120782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1659/2586] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:28 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:29.125075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:29.125672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:29.131924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.132364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.133099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.133525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.201579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:29.213971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d02e4ab0-f855-4a29-8fd3-d2706be37da7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.091979 Jul 20 15:24:29.214791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1660/2587] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:29 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:29.219322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:29.220309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:29.226119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.226569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.227349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.227781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.253259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:29.258284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9d601a68-3789-4cf6-beae-bbd2333b3e75 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042510 Jul 20 15:24:29.258828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1661/2588] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:29 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:29.970239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:29.971405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:29.978546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.978997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:29.979784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:29.980214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.041464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:30.049585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0bd06d88-f588-4e3e-bdda-6314e1b94a64 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.086456 Jul 20 15:24:30.050149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1662/2589] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:29 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:30.142253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:30.143299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:30.150431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.150881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.151664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.152095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.298732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:30.308260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:30.309743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:30.310867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b141f20e-cc6b-4e96-8673-c71fa8a3a4b7 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.174917 Jul 20 15:24:30.311457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1663/2590] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:30 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:30.315822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:30.316414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:30.322635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.323091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.323843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.324271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.389811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:30.398120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f1d0b47c-2391-4357-a883-79a9ce8688bf tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085706 Jul 20 15:24:30.398681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1664/2591] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:30 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:30.403348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:30.404285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:30.410291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.410740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.411565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:30.412009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:30.441067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cf59ddca-dd09-4eee-ab0e-99760775d08c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041383 Jul 20 15:24:30.441686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1665/2592] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:30 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:31.070893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:31.071962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:31.082714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.083200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.083943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.084396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.152158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:31.164266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2703682c-2ce3-4e9f-ae3f-7fff6cbead34 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.099905 Jul 20 15:24:31.165063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1666/2593] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:31 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:31.333612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:31.334490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:31.344745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.345494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.346741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.347522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.500363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:31.509102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:31.510581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:31.511765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3fa30f6e-c67a-49b8-8ec9-c72a524869d5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.184770 Jul 20 15:24:31.512351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1667/2594] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:31 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:31.516642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:31.517236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:31.523228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.523672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.524416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.524842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.586708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:31.599697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a583e695-11e3-498f-8ee5-fa25cc961847 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085988 Jul 20 15:24:31.600661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1668/2595] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:31 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:31.608309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:31.609817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:31.617845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.618287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.619038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:31.619485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:31.654045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d0d227f-e02d-4ab5-9adc-f98b4c204219 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052090 Jul 20 15:24:31.654604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1669/2596] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:31 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:32.184006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:32.184996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:32.193806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.194252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.195002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.195470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.259767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:32.267937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cebd42be-96b4-46a8-b19f-b4fa26f929a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.089596 Jul 20 15:24:32.268512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1670/2597] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:32 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:32.532468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:32.533315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:32.542068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.542691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.543794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.544401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.682920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:32.691936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:32.693439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:32.694609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6feeb09e-314c-49d1-a7b3-f26ffa98a8a5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167732 Jul 20 15:24:32.695194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1671/2598] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:32 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:32.699960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:32.700580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:32.707615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.708344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.709750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.710379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.780003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:32.788351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfe54fbe-5b2d-44a6-9be3-af0aa23b423d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.092140 Jul 20 15:24:32.788944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1672/2599] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:32 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:32.793376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:32.794283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:32.800404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.800885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.801642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:32.802071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:32.832384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87730615-cc41-4985-89cb-900684987407 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042415 Jul 20 15:24:32.832939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1673/2600] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:32 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:33.290887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:33.291691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:33.299076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.299572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.300399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.300987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.374507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:33.382638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-61fde565-ab42-4126-8896-d4242420b1be tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096141 Jul 20 15:24:33.383233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1674/2601] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:33 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:33.717257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:33.718253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:33.726493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.726940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.727731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.728155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.862862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:33.872062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:33.873598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:33.874845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4b8716ff-5a8f-431c-9c8e-8c27207a354a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.164075 Jul 20 15:24:33.875483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1675/2602] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:33 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:33.879884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:33.880490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:33.887015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.887501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.888258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.888692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.951434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:33.959551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a87f48c-32b1-4a08-ad7f-1098b6ba3d0b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.083135 Jul 20 15:24:33.960118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1676/2603] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:33 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:33.964555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:33.965463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:33.971466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.971848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:33.972627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:33.973056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:34.008989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7a4ae033-c785-4d91-bd7c-5ad25d770c56 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047518 Jul 20 15:24:34.009712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1677/2604] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:33 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:34.404097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:34.405151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:34.414034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:34.414483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:34.415282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:34.415686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:34.433825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:34.477867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:34.486525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fa415904-a792-4ae6-b93f-a9c8137bff37 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088432 Jul 20 15:24:34.486656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1678/2605] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:34 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:34.896315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:34.896945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:34.903340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:34.903793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:34.904547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:34.904982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.042788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:35.051932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:35.053643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:35.055499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba458651-111d-4266-b1de-b1187e3f1433 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.164665 Jul 20 15:24:35.056294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1679/2606] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:34 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:35.060641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:35.061266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:35.067573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.070422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.073396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.076048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.142222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:35.150452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b58ecb1c-2456-480a-a175-cc2575c5a03f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093222 Jul 20 15:24:35.151239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1680/2607] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:35 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:35.155777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:35.156699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:35.162495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.162942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.163726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.164157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.193222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-af37d436-af12-4a96-bbc6-7af048cdcf0b tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041091 Jul 20 15:24:35.193768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1681/2608] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:35 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:35.504195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:35.504794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:35.511911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.512361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.513169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:35.513602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:35.579808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:35.588502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce03f640-ea99-4c61-b038-421f89bb2e14 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.088155 Jul 20 15:24:35.589075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1682/2609] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:35 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:24:36.078432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:36.079493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:36.086928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.087409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.088167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.088597 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.230346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:36.239645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:36.241131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:36.242294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-187fa6d4-1d33-40f1-a7cb-d25fd08ef23c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170309 Jul 20 15:24:36.242851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1683/2610] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:36 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:36.247113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:36.247747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:36.253568 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.254092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.254885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.255351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.322046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:36.330311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9821485e-fe18-4d99-a079-56a4da7e1901 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086434 Jul 20 15:24:36.330810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1684/2611] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:36 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:36.335373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:36.336281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:36.342225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.342661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.343474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.343902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.373302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-db0a7af2-e3ce-4f2e-af83-56235e0f3c02 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041440 Jul 20 15:24:36.373861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1685/2612] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:36 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:36.609639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:36.610629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:36.622218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.622955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.624247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:36.624959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:36.688040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:36.696765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3a8571f-3c2d-408f-9c2f-d369f4710d1e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.093649 Jul 20 15:24:36.697339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1686/2613] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:36 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:37.264811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:37.265810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:37.276331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.277089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.278339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.279099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.414320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:37.422906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:37.424423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:37.425620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1db3ecab-949a-4ce4-b7a2-3e4e16ebc226 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.167397 Jul 20 15:24:37.426193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1687/2614] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:37 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:37.430782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:37.431410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:37.437453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.437909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.438751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.439244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.502863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:37.511634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5267026d-d510-4d40-b050-1b7827b04b03 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084423 Jul 20 15:24:37.512205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1688/2615] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:37 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:37.516784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:37.517626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:37.523435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.523883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.524633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.525069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.554285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eac3980c-8489-4e13-9451-6ce51892ba0f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041110 Jul 20 15:24:37.554843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1689/2616] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:37 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:37.717462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:37.718502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:37.729578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.730330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.731651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:37.732378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:37.798548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:37.807636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16177460-202e-4b78-b02f-cca0f30248a3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.096496 Jul 20 15:24:37.808187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1690/2617] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:37 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:38.448207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:38.449213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:38.458680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.459176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.459935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.460367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.594458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:38.608850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:38.611328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:38.613542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d066ce50-2b49-4c69-ae92-92f5d3a01e1f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.171787 Jul 20 15:24:38.614108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1691/2618] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:38 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:38.618453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:38.619027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:38.624994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.625443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.626204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.626633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.691854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:38.699933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c5b2c73-1f30-4fac-81ff-0fbd2e1f180d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084813 Jul 20 15:24:38.700463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1692/2619] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:38 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:38.705007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:38.705920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:38.711866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.712366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.713127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.713613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.743408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dca2e99e-d08f-4348-9852-6eaebe16da44 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041885 Jul 20 15:24:38.744110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1693/2620] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:38 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:38.824758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:38.825356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:38.831996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.832504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.833259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:38.833685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:38.904021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:38.912258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e60096bc-54d2-4b55-bb74-b7d4b4634fee tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.091389 Jul 20 15:24:38.912839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1694/2621] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:38 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:39.636976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:39.638091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:39.649155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.649918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.651302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.652112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.659019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:39.805122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:39.817280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:39.818749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:39.819904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-65b07480-5b6a-4f95-b09e-74125434b745 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.189876 Jul 20 15:24:39.820495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1695/2622] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:39 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:39.824835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:39.825432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:39.826980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:39.832833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.833270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.834154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.834580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.906241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:39.914577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d20b9c68-3d99-4ace-9e1d-1c4d51c911a3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.093086 Jul 20 15:24:39.915086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1696/2623] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:39 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:39.919731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:39.920682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:39.926752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.927219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.928079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.928486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.957825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0807edf8-71b3-4191-992c-11484d94fecd tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041745 Jul 20 15:24:39.958353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1697/2624] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:39 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:39.962891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:39.963511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:39.969018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.969463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:39.970196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:39.970623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:40.030866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:40.039427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a5c5445-860f-451d-ba7e-07e03c0a13aa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.079973 Jul 20 15:24:40.039985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1698/2625] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:39 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:40.840661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:40.841518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:40.851122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:40.851861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:40.853023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:40.853652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:40.995218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:41.003881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:41.005385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:41.006579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4a95e76d-b8b8-447b-bad9-dce454552c5b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.171166 Jul 20 15:24:41.007181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1699/2626] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:40 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:41.011574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:41.012180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:41.018274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.018751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.019532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.019969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.085488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:41.093865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fbc04eea-0101-4bac-8b03-6540f78282a0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.085639 Jul 20 15:24:41.094359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1700/2627] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:41 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:41.099166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:41.100069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:41.105881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.106326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.107084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.107519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.137290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2c2977ff-99e5-49c4-864e-fc337b87a0c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041924 Jul 20 15:24:41.137841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1701/2628] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:41 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:41.142061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:41.142655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:41.148383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.148841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.149578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:41.150006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:41.211705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:41.220061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2b05e46f-43ed-4097-b4b1-edca1998602b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.081295 Jul 20 15:24:41.220645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1702/2629] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:41 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:42.028557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:42.029648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:42.038612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.039073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.039840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.040272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.180790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:42.189419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:42.190908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:42.192133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b322dc0e-f898-46d3-a436-a32b7479d142 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170252 Jul 20 15:24:42.192700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1703/2630] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:42 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:42.197041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:42.197640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:42.203862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.204308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.205064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.205496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.266232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:42.274476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-db868a88-6880-45e9-b93f-e797cb405e38 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.080819 Jul 20 15:24:42.275059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1704/2631] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:42 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:42.279536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:42.280379 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:42.286020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.286464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.287250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.287681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.316533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e8212982-4074-4977-afaa-7666cbe9f895 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040491 Jul 20 15:24:42.317096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1705/2632] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:42 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:42.321351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:42.321948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:42.327501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.327942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.328680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:42.329107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:42.395354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:42.403986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-43ef0ad3-9b46-446e-ba3e-253a10d9869f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.085954 Jul 20 15:24:42.404556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1706/2633] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:43.216694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:43.217833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:43.226572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.227017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.227792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.228216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.369600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:43.378174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:43.379777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:43.381052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b5c70831-9829-424c-b309-83bc6e1cb1d3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.170911 Jul 20 15:24:43.381815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1707/2634] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:43 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:43.386098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:43.386669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:43.393013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.393455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.394204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.394628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.457080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:43.465511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6dad551c-48ce-4f58-a907-a650f91366f5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.082900 Jul 20 15:24:43.466082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1708/2635] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:43 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:43.470588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:43.471474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:43.477450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.477891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.478643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.479088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.508929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-14083fa6-badf-40f8-a367-82bdc20ebd61 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041819 Jul 20 15:24:43.509480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1709/2636] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:43 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:43.513695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:43.514288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:43.520105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.520539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.521401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:43.521824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:43.586537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:43.598184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8042e65e-7ea2-412f-b4a1-893b2c1e894a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087580 Jul 20 15:24:43.598800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1710/2637] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:43 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:44.404017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:44.405116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:44.416320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.417022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.418294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.419000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.574448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:44.578048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:44.584667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:44.586146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:44.587459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bf4d057d-b277-44c0-8f5b-b7198040ede5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.190081 Jul 20 15:24:44.588020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1711/2638] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:44 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:44.592321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:44.592917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:44.598872 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.599359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.600112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.600544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.666809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:44.675037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32f4574e-6421-4725-9b34-19f196d850fe tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086068 Jul 20 15:24:44.675702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1712/2639] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:44 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:44.680118 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:44.681015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:44.686556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.687003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.687897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.688331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.716184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-406591a2-630c-4392-8896-7bc448620be8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.039488 Jul 20 15:24:44.716727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1713/2640] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:44 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:44.720918 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:44.721510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:44.727854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.728343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.729081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:44.729507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:44.791365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:44.799742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e6a9ec4b-0fb7-4835-88fd-0bc5eeff77eb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082101 Jul 20 15:24:44.800311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1714/2641] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:44 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:45.610153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:45.611201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:45.622687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.623496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.624744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.625478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.765276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:45.774007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:45.775532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:45.776666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6302994e-b4a1-46cc-a0b6-fa0678d650cc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.172958 Jul 20 15:24:45.777221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1715/2642] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:45 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:45.781642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:45.782240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:45.788238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.788688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.789442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.789873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.852474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:45.864362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0485e91-28c3-48bd-b3c4-7df606fc844c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.086199 Jul 20 15:24:45.864933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1716/2643] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:45 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:45.869375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:45.870286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:45.875992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.876439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.877186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.877617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.906090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5806ab7e-233a-45fc-a0e1-fd1e1ca7812e tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040131 Jul 20 15:24:45.906622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1717/2644] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:45 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:45.910823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:45.911537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:45.917529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.917967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.918713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:45.919195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:45.996092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:46.008730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0cded504-844a-4c04-9107-f6426140d80a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.100866 Jul 20 15:24:46.009651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1718/2645] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:45 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:46.794891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:46.795549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:46.802534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:46.802983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:46.803769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:46.804203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:46.950995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:46.965469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:46.968002 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:46.969839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76a14cec-8911-4e77-b11e-24a8c2671b1c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.178610 Jul 20 15:24:46.970602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1719/2646] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:46 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:46.975008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:46.975631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:46.981468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:46.981920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:46.982674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:46.983144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:47.047650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:47.056114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7660c35e-bc5c-4bfe-a2ec-f95f344f00c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1369 microversion: 2.1 time: 0.084531 Jul 20 15:24:47.056663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1720/2647] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:46 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:47.061200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:47.062108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:47.067914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:47.068362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:47.069150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:47.069582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:47.098744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4bd95e9d-dffc-4717-abc0-3e6589b5c973 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040990 Jul 20 15:24:47.099344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1721/2648] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:47 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:47.103643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:47.104243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:47.110072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:47.110462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:47.111244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:47.111782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:47.174456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:47.182700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ae5260f-40d1-4910-9126-ae56b58ecd2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.082371 Jul 20 15:24:47.183302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1722/2649] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:47 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:47.989711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:47.990854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:48.002340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.002925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.003782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.004192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.149750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:48.162086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:48.163705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:48.165476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2f2a055f-a63a-4527-9dab-be77de7b9c0b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1802 microversion: 2.20 time: 0.181047 Jul 20 15:24:48.166256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1723/2650] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:47 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1802 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:24:48.170540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:48.171143 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:48.176969 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.177414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.178163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.178598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.208540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.208888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.209331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'fault' on Instance uuid 52220445-7cc1-4635-9566-c6fd58b1fc96 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:48.256459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:48.264524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e2612fdc-d901-41d5-8aee-d299754d8dbd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1463 microversion: 2.1 time: 0.097340 Jul 20 15:24:48.265093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1724/2651] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:48 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1463 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:48.269747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:48.270675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:48.276353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.276796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.277571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.277998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.307402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-51d17848-6198-431b-88bd-68a13bf00266 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041242 Jul 20 15:24:48.307938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1725/2652] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:48 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:48.312128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:48.312716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:48.319092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.319572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.320377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:48.320812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:48.386971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:48.396252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c0be598d-9608-4612-9eba-0fd2dbacdbb6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1372 microversion: 2.1 time: 0.087293 Jul 20 15:24:48.396809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1726/2653] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:48 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1372 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:49.188010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:49.189067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:49.196814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.197259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.198018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.198452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.348513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:49.357518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:49.359008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:49.360214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0fdefca2-a5ce-4e0f-8ba2-70982639a385 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.178649 Jul 20 15:24:49.360778 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1727/2654] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:49 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:49.365479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:49.366078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:49.372149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.372606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.373368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.373805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.377929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:49.405284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.405728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.406166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'fault' on Instance uuid 52220445-7cc1-4635-9566-c6fd58b1fc96 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:49.447606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:49.455811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5da0c740-7e88-4432-b50a-654fc19fe370 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1463 microversion: 2.1 time: 0.094095 Jul 20 15:24:49.456400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1728/2655] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:49 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1463 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:49.460836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:49.461752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:49.467712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.468164 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.468914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.469345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.499696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a74bb811-9f8b-4392-90e4-36ea4291d7f8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042293 Jul 20 15:24:49.500275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1729/2656] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:49 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:49.506493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:49.507082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:49.512993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.513439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.514185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.514614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.544418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:49.544862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:49.545306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:49.589709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:49.597892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f27d570-4c1e-480d-9ed7-b9fec1170abc tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1466 microversion: 2.1 time: 0.096703 Jul 20 15:24:49.598492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1730/2657] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:49 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1466 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:50.382238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:50.383324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:50.394249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.394726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.395501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.395956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.538194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:50.546760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:50.548298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:50.549461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b850e83-b1ab-422a-8e60-c67c704b87a4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.173851 Jul 20 15:24:50.550037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1731/2658] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:50 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:50.554742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:50.555401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:50.561398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.561851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.562609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.563098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.594794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.595346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.595814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'fault' on Instance uuid 52220445-7cc1-4635-9566-c6fd58b1fc96 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:50.638167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:50.646239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8f1c5a0c-961c-419d-a478-f67991257492 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1463 microversion: 2.1 time: 0.095134 Jul 20 15:24:50.646937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1732/2659] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:50 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1463 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:50.651427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:50.652354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:50.658141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.658573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.659357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.659823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.689523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-812fae08-2b77-4e0a-a6d6-55ff8475bc8f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041561 Jul 20 15:24:50.690050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1733/2660] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:50 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:50.694380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:50.695018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:50.700712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.701159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.701904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.702332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.732729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:50.733171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:50.733614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:50.781003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:50.790912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ec09db95-2de6-4258-95ea-2ed94138b131 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1466 microversion: 2.1 time: 0.099909 Jul 20 15:24:50.791508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1734/2661] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:50 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1466 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:51.571200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:51.572018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:51.581144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.581736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.582718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.583588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.716040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:51.724963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:51.726506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:51.727699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8c292080-931b-42c6-9234-07420bb44ae6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.162300 Jul 20 15:24:51.728256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1735/2662] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:51 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:51.732511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:51.733105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:51.739699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.740116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.740852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.741274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.771590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.772025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.772449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'fault' on Instance uuid 52220445-7cc1-4635-9566-c6fd58b1fc96 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:51.818293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:51.826801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9ebbc103-1cfb-49df-82d1-cb45ab4f9d41 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 200 len: 1463 microversion: 2.1 time: 0.097628 Jul 20 15:24:51.827594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1736/2663] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:51 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 1463 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:51.832116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:51.833050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:51.839325 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.839807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.840623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.841091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.873117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe7378ac-8347-4955-98a2-bd841e616d06 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044621 Jul 20 15:24:51.873696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1737/2664] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:51 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:51.878354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:51.878957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:51.885127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.885687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.886445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.886870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.917931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:51.918342 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:51.918782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 6b566c6c-a418-4384-b99e-c7295e754e92 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:24:51.961641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:51.969839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4ef519b2-8495-4f1f-a793-3cabf2a0f771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 200 len: 1466 microversion: 2.1 time: 0.095136 Jul 20 15:24:51.970396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1738/2665] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:51 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 1466 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:24:52.747114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:52.747779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:52.750050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:52.756090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.756540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.757381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.757837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.908417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:52.916977 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:52.918470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:52.920030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-25371644-88fc-4563-bbad-775eb0d853db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.177086 Jul 20 15:24:52.920603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1739/2666] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:52 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:52.925251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:52.925853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:52.932758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.933215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.934056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.934490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.948261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 HTTP/1.1 Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:24:52.949919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:24:52.951775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '52220445-7cc1-4635-9566-c6fd58b1fc96'} Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:24:52.952881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:24:52.954340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:52.955680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:24:52.955680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0511e88f-7a99-4e3d-96b0-dade8e0f222c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 404 len: 111 microversion: 2.1 time: 0.029768 Jul 20 15:24:52.955680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1740/2667] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:52 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:24:52.956516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:52.957434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:52.963291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.963740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.964492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:52.964925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:52.995511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87c68de6-e446-4499-8e5a-1891b9d79cba tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042654 Jul 20 15:24:52.996237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1741/2668] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:24:52 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:24:53.000585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cbd40954-b774-4ceb-9cea-3fc6c7be5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '.version_select at 0x7f7b15de5a60>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:53.022833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cbd40954-b774-4ceb-9cea-3fc6c7be5d10 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2080322666" status: 202 len: 0 microversion: 2.1 time: 0.025647 Jul 20 15:24:53.023442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1742/2669] 10.222.0.15 () {58 vars in 1220 bytes} [Wed Jul 20 15:24:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2080322666 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:24:53.027878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:53.028488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:53.034585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:53.035036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:53.035859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:53.036294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:53.051686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:24:53.051925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:24:53 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:24:53.056175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 HTTP/1.1 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:24:53.057823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:24:53.059489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '6b566c6c-a418-4384-b99e-c7295e754e92'} Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:24:53.060375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:24:53.062637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:24:53.065251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:24:53.065251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-99e40401-8a6a-452b-b8d3-9073408b4120 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 404 len: 111 microversion: 2.1 time: 0.034951 Jul 20 15:24:53.065251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1743/2670] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:24:53 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:24:53.070579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea323f0b-47e2-4703-aa6e-04b44c156c02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '.version_select at 0x7f7b1ad2b310>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:53.090536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea323f0b-47e2-4703-aa6e-04b44c156c02 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1075877835" status: 202 len: 0 microversion: 2.1 time: 0.023788 Jul 20 15:24:53.091119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1744/2671] 10.222.0.15 () {58 vars in 1220 bytes} [Wed Jul 20 15:24:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1075877835 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:24:53.786485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-97c9ad76-1af2-4fd9-bdbc-11474bf3ac62 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: .version_select at 0x7f7b15bee670>, body: {"keypair": {"name": "tempest-keypair-179996137", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMjDILq+9yu2LZCIj/zbbMK0d9Q+whYOqBw7nZ5o8rMhYQeK27w1cqgN9pN0YQ9hBHkdd++uQ3x2cPt2ITrs2j+asMj55XZ2ro7YfVW2+lrkdn4scfLaE45Sh0ccsnoAcA=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:53.804732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-97c9ad76-1af2-4fd9-bdbc-11474bf3ac62 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for project b966dbac18614c979f674666ba817def. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:24:53.808600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-97c9ad76-1af2-4fd9-bdbc-11474bf3ac62 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for user 5f51124af1e846fb89552078c81afdb0 and project b966dbac18614c979f674666ba817def. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:24:53.831857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-97c9ad76-1af2-4fd9-bdbc-11474bf3ac62 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.049167 Jul 20 15:24:53.832457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1745/2672] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 15:24:53 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:24:53.903802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e72b1ebe-432f-4277-a389-4b2333d43d6f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: .version_select at 0x7f7b15ec5b80>, body: {"keypair": {"name": "tempest-keypair-341583278", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA9QZQNF2G7m+cqrLK4aZ8Yf+vTZ7laSVBbEnkMhCw+saY93uL2fVe0IyoMHub4KH/RtygCkQzzW+NpCyttsPMXX/tGHuaZy/78Q9W0uNm08fl324TEFPhEva1bKElwyGA=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:53.919554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-e72b1ebe-432f-4277-a389-4b2333d43d6f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:24:53.923801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-e72b1ebe-432f-4277-a389-4b2333d43d6f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:24:53.948196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e72b1ebe-432f-4277-a389-4b2333d43d6f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.049345 Jul 20 15:24:53.949106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1746/2673] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 15:24:53 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:24:53.953674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:53.954283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:53.960390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:53.960840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:53.961595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:53.962537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:54.190090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:54.198513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:54.200026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:54.201193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9dc20efb-038c-4d24-b8cc-478a5bf105e4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.251144 Jul 20 15:24:54.201756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1747/2674] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:53 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:54.206595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:54.207521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:54.213500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:54.213941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:54.214808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:54.215297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:54.219533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:24:56.937124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8c0f0b73-1ce1-4af4-9715-409177347422 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action" status: 200 len: 118396 microversion: 2.1 time: 2.733653 Jul 20 15:24:56.938253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1748/2675] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:24:54 2022] POST /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8/action => generated 118396 bytes in 2736 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:56.944602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:24:56.945322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:24:56.953053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:56.953642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:56.954392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:56.954949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:57.090490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:57.098467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:57.099979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:24:57.101144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32a34d61-75ad-4ec6-b905-cd95adad51c0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 200 len: 1724 microversion: 2.20 time: 0.161496 Jul 20 15:24:57.101713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1749/2676] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:24:56 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:24:57.106531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1332863119", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "696bc9e1-cb60-427e-98c0-b21d1bdfb8d1"}], "security_groups": [{"name": "tempest-securitygroup--408075937"}], "key_name": "tempest-keypair-179996137", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:57.268535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] validate_networks() for [('696bc9e1-cb60-427e-98c0-b21d1bdfb8d1', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:24:58.250549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:24:58.251192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:24:58.252234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:24:58.252764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:24:58.253522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:24:58.263703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:58.264026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:58.264807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:58.265275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:58.284803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for project b966dbac18614c979f674666ba817def. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:24:58.288992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Getting quotas for user 5f51124af1e846fb89552078c81afdb0 and project b966dbac18614c979f674666ba817def. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:24:58.300916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:24:58.303249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:24:58.356533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-19d71080-a069-41b7-a23d-2648ca4e7350 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.253707 Jul 20 15:24:58.357346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1750/2677] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 15:24:57 2022] POST /compute/v2.1/servers => generated 401 bytes in 1255 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 15:24:58.366999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-80677610", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "bb36c589-f60d-4c91-8345-5777287e4f0d"}], "security_groups": [{"name": "tempest-securitygroup--1695827965"}], "key_name": "tempest-keypair-341583278", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:58.566902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] validate_networks() for [('bb36c589-f60d-4c91-8345-5777287e4f0d', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:24:59.064411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:24:59.064828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:24:59.065290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:24:59.065802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:24:59.066215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:24:59.075139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:59.075609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:59.076408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:59.077030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:59.095214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:24:59.099110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:24:59.112299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:24:59.115150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:24:59.178967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6adb9d38-4b8d-46e9-8553-dee178136039 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.817497 Jul 20 15:24:59.186422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1751/2678] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 15:24:58 2022] POST /compute/v2.1/servers => generated 402 bytes in 828 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 15:24:59.194517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1821432340", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "networks": [{"uuid": "3e43663e-3658-4f8c-a038-ce7c5d38739c"}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:24:59.296765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] validate_networks() for [('3e43663e-3658-4f8c-a038-ce7c5d38739c', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:24:59.955898 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:24:59.956903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:24:59.957681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:24:59.958761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:24:59.959526 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:24:59.969773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:59.970395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:59.971497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:24:59.972185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:24:59.992461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Getting quotas for project 1cdab962e8b0462b8d1d28dcde703c15. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:24:59.998337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Getting quotas for user bc2f1a0d431341d6a5d1a8c5de131ece and project 1cdab962e8b0462b8d1d28dcde703c15. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:25:00.012241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:25:00.014773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:25:00.121683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-385667a0-f4f0-4f68-979f-864d5557d55a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.933236 Jul 20 15:25:00.128775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1752/2679] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 15:24:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 941 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 15:25:00.138046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.139123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.147003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.147770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.148838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.149571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.170450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d HTTP/1.1 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:25:00.172350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:25:00.174059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d'} Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:25:00.175704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:25:00.177142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:00.179343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:25:00.180039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8dad3323-0d7f-4a0b-ad1e-2d339a6e27b5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 404 len: 111 microversion: 2.20 time: 0.050132 Jul 20 15:25:00.184371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1753/2680] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:00 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:25:00.187930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.188545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.190732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=87,nova_api:DELETE=2,nova_api:INSERT=11 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:00.197079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.197524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.198302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.198734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.317549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:00.327294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d0a946ba-f9ab-4b0a-a9b9-a1537f490184 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1307 microversion: 2.1 time: 0.145446 Jul 20 15:25:00.327943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1754/2681] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:00 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1307 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:00.332491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-3c7c1ce4-02fa-4dd9-8c4a-4f5eae9f51f3 req-f553551f-d138-4794-addc-763002a7b9db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3dfe0713-c262-4daa-8e0d-d439f6e68f7d", "name": "network-vif-deleted", "tag": "4274a4e2-f453-4d38-87d5-07ccb98f56b7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:00.340509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3c7c1ce4-02fa-4dd9-8c4a-4f5eae9f51f3 req-f553551f-d138-4794-addc-763002a7b9db service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.341028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3c7c1ce4-02fa-4dd9-8c4a-4f5eae9f51f3 req-f553551f-d138-4794-addc-763002a7b9db service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.352861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-3c7c1ce4-02fa-4dd9-8c4a-4f5eae9f51f3 req-f553551f-d138-4794-addc-763002a7b9db service nova] Creating event network-vif-deleted:4274a4e2-f453-4d38-87d5-07ccb98f56b7 for instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d on hv-cinder-84902 Jul 20 15:25:00.358168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-3c7c1ce4-02fa-4dd9-8c4a-4f5eae9f51f3 req-f553551f-d138-4794-addc-763002a7b9db service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029034 Jul 20 15:25:00.358783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1755/2682] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:00.367452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.368306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.375225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.375695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.376729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.377249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.447770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:00.455711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-16fff719-71e8-4177-a192-bded57a1b9ad tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1291 microversion: 2.1 time: 0.092329 Jul 20 15:25:00.456481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1756/2683] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:00 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1291 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:00.461907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.462471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.468792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.469248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.470160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.470588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.498665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:25:00.595993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3e90654f-771f-4fb1-b93f-91228bb67e32 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 204 len: 0 microversion: 2.1 time: 0.138060 Jul 20 15:25:00.596679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1757/2684] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:25:00 2022] DELETE /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:25:00.602775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.603535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.612834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.613448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.614367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.615385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.679787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:00.687864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a4ce6f6c-7a44-4c7c-9ca4-3e7aecebd1ec tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1223 microversion: 2.1 time: 0.089579 Jul 20 15:25:00.688482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1758/2685] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:00 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1223 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:00.702923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e6a2862-6534-4992-9c2e-69c2794e64da tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '.version_select at 0x7f7b15b26820>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.721069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e6a2862-6534-4992-9c2e-69c2794e64da tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1676120478" status: 204 len: 0 microversion: 2.20 time: 0.022038 Jul 20 15:25:00.721889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1759/2686] 10.222.0.15 () {60 vars in 1261 bytes} [Wed Jul 20 15:25:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1676120478 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 20 15:25:00.727041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:00.727745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:00.734018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.734869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.735436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:00.737281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:00.922428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:00.930490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f3bda55d-0833-4096-af4e-1d3b35b8ac65 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1689 microversion: 2.1 time: 0.207147 Jul 20 15:25:00.931098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1760/2687] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:00 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1689 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:01.345005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:01.345661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:01.352714 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.353214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.354296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.354734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.626345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:01.641698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e485a269-0bd9-4631-9680-041c520a889c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.300813 Jul 20 15:25:01.642287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1761/2688] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:01 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:01.662809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:01.668065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:01.686126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.686806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.687871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.688459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.777892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:01.785629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea0bdd41-dcac-45d3-94a7-fbfe856b7b20 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1309 microversion: 2.1 time: 0.133821 Jul 20 15:25:01.786204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1762/2689] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:01 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1309 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:01.790552 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:01.791180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:01.797930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.798365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.799156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.799588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.875783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:01.884150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-21455a83-229a-4a6f-92af-34eac9ba3e50 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1271 microversion: 2.1 time: 0.096914 Jul 20 15:25:01.884692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1763/2690] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:01 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1271 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:01.889518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3b362768-4649-4279-809e-3e950ed4e7b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: .version_select at 0x7f7b1ad6d310>, body: {"keypair": {"name": "tempest-keypair-2021460468", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBH4d89VMjwNWmsxOX/UsoGE1/bdjjucW8Ylzpm7jiJV4xsd48EVuTiC56Avd3IZmbvEeHrxJBZnUafc2aQhK+AnsD64fe3r6YBFqRTuLUcR38GalopAqnUu76z5nE0Je6w=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:01.904110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-3b362768-4649-4279-809e-3e950ed4e7b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:25:01.907953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-3b362768-4649-4279-809e-3e950ed4e7b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for user ee5a234904e64eb9b2faad85909d0507 and project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:25:01.931007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3b362768-4649-4279-809e-3e950ed4e7b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.045105 Jul 20 15:25:01.931941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1764/2691] 10.222.0.15 () {62 vars in 1199 bytes} [Wed Jul 20 15:25:01 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 47 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 20 15:25:01.948229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:01.948695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:01.955623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.956211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:01.956917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:01.957481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:02.383643 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:02.396144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-76ab4c87-cfff-4061-b215-64d79706588d tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1689 microversion: 2.1 time: 0.450606 Jul 20 15:25:02.398865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1765/2692] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:01 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1689 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:25:02.506553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-d164d5fa-1f2e-4e4f-86b9-94327de85744 req-119464bf-50ca-4cbe-a05a-2fdc446a80d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae", "tag": "b9ddd43f-d051-4dea-8402-f48e7e16421e"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:02.518575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d164d5fa-1f2e-4e4f-86b9-94327de85744 req-119464bf-50ca-4cbe-a05a-2fdc446a80d9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:02.519089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d164d5fa-1f2e-4e4f-86b9-94327de85744 req-119464bf-50ca-4cbe-a05a-2fdc446a80d9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:02.534066 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-d164d5fa-1f2e-4e4f-86b9-94327de85744 req-119464bf-50ca-4cbe-a05a-2fdc446a80d9 service nova] Creating event network-changed:b9ddd43f-d051-4dea-8402-f48e7e16421e for instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae on hv-cinder-84902 Jul 20 15:25:02.540277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-d164d5fa-1f2e-4e4f-86b9-94327de85744 req-119464bf-50ca-4cbe-a05a-2fdc446a80d9 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.138704 Jul 20 15:25:02.541504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1766/2693] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:02.547766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-bd481c78-7e25-4781-8f97-99a9121d8626 req-c7425597-6285-43a9-8362-d80549271ab2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e827c810-220b-4fe1-9d16-01bc8372b933", "name": "network-vif-deleted", "tag": "1f670f13-f9d2-4b6a-b0ce-7144e9adc59f"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:02.557851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-bd481c78-7e25-4781-8f97-99a9121d8626 req-c7425597-6285-43a9-8362-d80549271ab2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:02.558499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-bd481c78-7e25-4781-8f97-99a9121d8626 req-c7425597-6285-43a9-8362-d80549271ab2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:02.571065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-bd481c78-7e25-4781-8f97-99a9121d8626 req-c7425597-6285-43a9-8362-d80549271ab2 service nova] Creating event network-vif-deleted:1f670f13-f9d2-4b6a-b0ce-7144e9adc59f for instance e827c810-220b-4fe1-9d16-01bc8372b933 on hv-cinder-84902 Jul 20 15:25:02.577152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-bd481c78-7e25-4781-8f97-99a9121d8626 req-c7425597-6285-43a9-8362-d80549271ab2 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033799 Jul 20 15:25:02.577886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1767/2694] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:02.662416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:02.663284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:02.670183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:02.670718 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:02.671653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:02.672086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.349920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:03.358548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-afd9c6bd-bb60-42e5-a8e4-40e695f4e8f0 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.700406 Jul 20 15:25:03.368776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1768/2695] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:02 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:03.374168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:03.374761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:03.386916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.387689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.394960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.395536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.487198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:03.496088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dc5204b0-8ca7-4e5a-8b59-b7e8880090ed tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1297 microversion: 2.1 time: 0.125707 Jul 20 15:25:03.496681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1769/2696] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:03 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1297 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:03.511337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:03.511959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:03.518949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.519758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.520507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.521119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.527702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:03.627932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:03.637297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6b9e3908-2220-4a6d-85f6-82f64bdd58fc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1277 microversion: 2.1 time: 0.129522 Jul 20 15:25:03.647165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1770/2697] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:03 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1277 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:03.647165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:03.647165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:03.652721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.653611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.654696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:03.655182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:03.983843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:04.002385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3ebd8b8f-cb03-47eb-be5a-aa20b2e0799f tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1689 microversion: 2.1 time: 0.362465 Jul 20 15:25:04.004560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1771/2698] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:03 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1689 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:04.376271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:04.379132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:04.391909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.393626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.394647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.396223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.571559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:04.579723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7d703766-37c7-4d20-838e-e1b5718bdee5 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.207251 Jul 20 15:25:04.580315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1772/2699] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:04 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:04.589218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:04.589835 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:04.596059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.596721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.597493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.598391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.742287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:04.750189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-38847e50-3365-42d4-a453-93d93a68b3ce tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.168893 Jul 20 15:25:04.759997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1773/2700] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:04 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:04.761358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-2af26daa-ea20-445e-86a8-5366dde8c6bd req-8ef5eb9a-7904-432e-90ef-12e683f440a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "75115162-3a7f-4a94-a703-190f8fdbc4c8", "name": "network-vif-deleted", "tag": "94ca1675-607f-45ff-a048-c7df5f9c493a"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:04.771432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2af26daa-ea20-445e-86a8-5366dde8c6bd req-8ef5eb9a-7904-432e-90ef-12e683f440a2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.771892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2af26daa-ea20-445e-86a8-5366dde8c6bd req-8ef5eb9a-7904-432e-90ef-12e683f440a2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.786095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-2af26daa-ea20-445e-86a8-5366dde8c6bd req-8ef5eb9a-7904-432e-90ef-12e683f440a2 service nova] Creating event network-vif-deleted:94ca1675-607f-45ff-a048-c7df5f9c493a for instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 on hv-cinder-84902 Jul 20 15:25:04.791228 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-2af26daa-ea20-445e-86a8-5366dde8c6bd req-8ef5eb9a-7904-432e-90ef-12e683f440a2 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034593 Jul 20 15:25:04.792028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1774/2701] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:04.800553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:04.801257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:04.807651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.808101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.808865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:04.809295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:04.957177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:04.964734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5df7e801-0a92-4543-87ea-3e2dcbc5e42f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.169125 Jul 20 15:25:04.965300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1775/2702] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:04 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:04.970199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1495366202", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4a7c71d7-d29f-4122-81d5-864e01fddc6d"}], "security_groups": [{"name": "tempest-securitygroup--333286395"}], "key_name": "tempest-keypair-2021460468", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:05.070878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] validate_networks() for [('4a7c71d7-d29f-4122-81d5-864e01fddc6d', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:25:05.464655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:25:05.465291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:25:05.465566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:25:05.465973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:25:05.466400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:25:05.476271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.477112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.477982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.478403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.500946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:25:05.505268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Getting quotas for user ee5a234904e64eb9b2faad85909d0507 and project 7db1a5ab36714ee2990b9c6c10ba0aea. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:25:05.517509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:25:05.519875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:25:05.566135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-968edc8f-490a-4e7f-867b-71f37278524d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.599787 Jul 20 15:25:05.567180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1776/2703] 10.222.0.15 () {62 vars in 1187 bytes} [Wed Jul 20 15:25:04 2022] POST /compute/v2.1/servers => generated 401 bytes in 601 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 20 15:25:05.571802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:05.572421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:05.579302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.579762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.580523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.580957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.647419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:05.655240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74d065c8-eb97-4263-8f5c-3f1135c70125 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1621 microversion: 2.1 time: 0.086962 Jul 20 15:25:05.655795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1777/2704] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:05 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1621 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:05.660870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:05.661479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:05.714131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:05.723274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:05.724773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:05.725915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-82a150d1-c766-4179-a911-0b262a1c3341 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1282 microversion: 2.20 time: 0.068968 Jul 20 15:25:05.726573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1778/2705] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:05 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1282 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:05.730904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:05.731604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:05.738123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.738750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.739541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.739970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.883015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:05.890578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba593e11-7dba-4de5-813e-7bb07c47c445 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.162983 Jul 20 15:25:05.891363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1779/2706] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:05 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:05.895583 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:05.896169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:05.902604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.903084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:05.903842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:05.904307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.039371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.050704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-50d7b5da-1b8d-4e2d-973f-535b27a2e419 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.158271 Jul 20 15:25:06.051530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1780/2707] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:05 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:06.055895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:06.056490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:06.063175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.063792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.064938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.065588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.324786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.332933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6cccfd29-cb40-4bf3-9c20-8354d9730ebb tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.280319 Jul 20 15:25:06.333990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1781/2708] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:06 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:06.676852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:06.677579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:06.689364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.690326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.692222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.692939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.763010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.763951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.764790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'fault' on Instance uuid 75115162-3a7f-4a94-a703-190f8fdbc4c8 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:25:06.839986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.857163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-00ec43cf-0492-488b-9f43-d5ffeac8c3dc tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 200 len: 1335 microversion: 2.1 time: 0.186447 Jul 20 15:25:06.858147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1782/2709] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:06 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 1335 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:06.870807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:06.871923 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:06.880783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.881825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.882743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:06.883479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:06.967195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.979622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.981744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:06.983131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dad814eb-dc56-48b1-8390-d9addc833076 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1334 microversion: 2.20 time: 0.119501 Jul 20 15:25:06.983876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1783/2710] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:06 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1334 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:06.999946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-c45cfdf6-097b-4e0b-81f3-53dfb5efdf82 req-cded3b5e-c486-44c7-9aca-766438fa07ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a94cc383-ac2c-4750-96b4-93a6110f0203", "tag": "10fab4eb-f028-49c9-9cdf-35861d81feb7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:07.018604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-c45cfdf6-097b-4e0b-81f3-53dfb5efdf82 req-cded3b5e-c486-44c7-9aca-766438fa07ac service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.019636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-c45cfdf6-097b-4e0b-81f3-53dfb5efdf82 req-cded3b5e-c486-44c7-9aca-766438fa07ac service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.047971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-c45cfdf6-097b-4e0b-81f3-53dfb5efdf82 req-cded3b5e-c486-44c7-9aca-766438fa07ac service nova] Creating event network-changed:10fab4eb-f028-49c9-9cdf-35861d81feb7 for instance a94cc383-ac2c-4750-96b4-93a6110f0203 on hv-cinder-84902 Jul 20 15:25:07.057456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-c45cfdf6-097b-4e0b-81f3-53dfb5efdf82 req-cded3b5e-c486-44c7-9aca-766438fa07ac service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067240 Jul 20 15:25:07.058791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1784/2711] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:07.067753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:07.069709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:07.077540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:07.079803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.080610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.081543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.082049 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.425708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:07.448350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e29da4fc-b479-4148-9fd4-91518176b85b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.388073 Jul 20 15:25:07.448948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1785/2712] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:07 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:07.461876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:07.462942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:07.471410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.472209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.473145 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.473681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.720887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:07.729429 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e1afdf45-9983-412b-a3f6-5d5831e78732 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.279464 Jul 20 15:25:07.730375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1786/2713] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:07 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:07.735159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:07.735943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:07.742656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.743144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.743904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.744333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.879218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:07.886892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e96f14a5-4a1a-4b0f-92e0-5e3bd285facc tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.155536 Jul 20 15:25:07.887497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1787/2714] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:07 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:07.891880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-7f202619-4e97-4ec9-84a2-cdd01518ec0f req-a6cb8918-c2e6-48e9-bc2b-2600282b04c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbbcd35f-cd1f-4195-958f-1e9d1194aea0", "tag": "c05d9926-8cc3-4a5e-8246-68dd7a0f2770"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:07.900076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7f202619-4e97-4ec9-84a2-cdd01518ec0f req-a6cb8918-c2e6-48e9-bc2b-2600282b04c2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.900541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7f202619-4e97-4ec9-84a2-cdd01518ec0f req-a6cb8918-c2e6-48e9-bc2b-2600282b04c2 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.911548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-7f202619-4e97-4ec9-84a2-cdd01518ec0f req-a6cb8918-c2e6-48e9-bc2b-2600282b04c2 service nova] Creating event network-changed:c05d9926-8cc3-4a5e-8246-68dd7a0f2770 for instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 on hv-cinder-84902 Jul 20 15:25:07.916290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-7f202619-4e97-4ec9-84a2-cdd01518ec0f req-a6cb8918-c2e6-48e9-bc2b-2600282b04c2 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027689 Jul 20 15:25:07.917042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1788/2715] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:07.922346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:07.923106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 75115162-3a7f-4a94-a703-190f8fdbc4c8] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:07.929799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.930232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.930993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:07.931485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:07.945377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 HTTP/1.1 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:25:07.947040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. Jul 20 15:25:07.948700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '75115162-3a7f-4a94-a703-190f8fdbc4c8'} Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:25:07.949842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. Jul 20 15:25:07.951514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:07.953152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 404 to user: Instance 75115162-3a7f-4a94-a703-190f8fdbc4c8 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:25:07.953152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-62f56d6b-f58a-482d-bee0-c63989336e8c tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8" status: 404 len: 111 microversion: 2.1 time: 0.030103 Jul 20 15:25:07.953152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1789/2716] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:07 2022] GET /compute/v2.1/servers/75115162-3a7f-4a94-a703-190f8fdbc4c8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:25:07.958178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-68fe2082-f069-4164-9983-138e782bfefb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '.version_select at 0x7f7b1ad521f0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:07.976293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-68fe2082-f069-4164-9983-138e782bfefb tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-509396105" status: 202 len: 0 microversion: 2.1 time: 0.021595 Jul 20 15:25:07.976972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1790/2717] 10.222.0.15 () {58 vars in 1217 bytes} [Wed Jul 20 15:25:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-509396105 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:25:08.001938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:08.002557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:08.008780 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.009227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:08.009982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.010407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:08.090236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:08.094151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:08.100661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:08.102450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:08.103603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8644c386-7996-4d10-ac01-41fdd9cd9383 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1340 microversion: 2.20 time: 0.105700 Jul 20 15:25:08.104471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1791/2718] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:07 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1340 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:25:08.465577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:08.466115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:08.472610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.473251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:08.473916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.474589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:08.962998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:08.970927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a208124f-cc2c-443d-85dc-577f55eb6e83 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.509440 Jul 20 15:25:08.971703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1792/2719] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:08 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:25:08.976370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:08.976955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:08.983652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.984106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:08.984874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:08.985305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.130107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:09.137702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0e371bb-6323-4809-90f8-acdf04b38953 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.165142 Jul 20 15:25:09.138295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1793/2720] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:08 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:09.142626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:09.143260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:09.148929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.149375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.150102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.150536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.281098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:09.293012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8799e887-38c9-424d-ba76-cdb60aabc0d4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.153714 Jul 20 15:25:09.293687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1794/2721] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:09 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:09.298296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:09.298892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:09.305009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.305504 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.306209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.306632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.439732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:09.448054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:09.450134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:09.451408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da83fe19-63a6-4539-b00e-096de44d9ad0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.156705 Jul 20 15:25:09.452053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1795/2722] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:09 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:09.988820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:09.989558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:09.996291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.996734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:09.997491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:09.997928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.218237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.226640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e253b3f-080b-472a-910c-da2f2497db44 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.242066 Jul 20 15:25:10.227809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1796/2723] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:09 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:10.241824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:10.242458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:10.255312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.255312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.255312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.255312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.469115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.493372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c383dfa0-7e37-43e7-8dba-777537b5cac5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.262960 Jul 20 15:25:10.495094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1797/2724] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:10 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:10.501536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:10.502461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:10.511122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.512072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.513268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.513268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.684025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.695282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aa3fa594-329f-4a0d-abd7-16ea8557b79c tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.198368 Jul 20 15:25:10.707188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1798/2725] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:10 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:10.712235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:10.712860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:10.720275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.720745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.721556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.721993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.883384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.892940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.894484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:10.895834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f2ab4293-6ad9-4afe-9c75-3319d67d1942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.187608 Jul 20 15:25:10.896407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1799/2726] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:10 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:10.959725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-edefb7c3-475f-41e9-a511-f29e6a32f5d1 req-52770541-2b82-4d5c-895d-e0d6e0ccf9c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae", "name": "network-vif-plugged", "status": "completed", "tag": "b9ddd43f-d051-4dea-8402-f48e7e16421e"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:10.968406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-edefb7c3-475f-41e9-a511-f29e6a32f5d1 req-52770541-2b82-4d5c-895d-e0d6e0ccf9c7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:10.968839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-edefb7c3-475f-41e9-a511-f29e6a32f5d1 req-52770541-2b82-4d5c-895d-e0d6e0ccf9c7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:10.981151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-edefb7c3-475f-41e9-a511-f29e6a32f5d1 req-52770541-2b82-4d5c-895d-e0d6e0ccf9c7 service nova] Creating event network-vif-plugged:b9ddd43f-d051-4dea-8402-f48e7e16421e for instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae on hv-cinder-84902 Jul 20 15:25:10.986146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-edefb7c3-475f-41e9-a511-f29e6a32f5d1 req-52770541-2b82-4d5c-895d-e0d6e0ccf9c7 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088634 Jul 20 15:25:10.987097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1800/2727] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:10.992422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-767c872c-4ffb-4b9f-bfdd-9414fb424afe req-764d6f56-9bee-4e0f-8832-4bb335451e80 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:11.001220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-767c872c-4ffb-4b9f-bfdd-9414fb424afe req-764d6f56-9bee-4e0f-8832-4bb335451e80 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.001658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-767c872c-4ffb-4b9f-bfdd-9414fb424afe req-764d6f56-9bee-4e0f-8832-4bb335451e80 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.013267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-767c872c-4ffb-4b9f-bfdd-9414fb424afe req-764d6f56-9bee-4e0f-8832-4bb335451e80 service nova] Creating event network-changed:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:25:11.017921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-767c872c-4ffb-4b9f-bfdd-9414fb424afe req-764d6f56-9bee-4e0f-8832-4bb335451e80 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029442 Jul 20 15:25:11.018555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1801/2728] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:11.245008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:11.245655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:11.252642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.253191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.253948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.254383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.388361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:11.396752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-544a85e8-2f40-4fa9-a786-62d59ce31715 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.156283 Jul 20 15:25:11.397303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1802/2729] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:11 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:11.401865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:11.402358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:11.408246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.408699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.409455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.409885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.437363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:25:11.521569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0df8b147-b373-4ed4-8c52-af2e0b2b9c69 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 204 len: 0 microversion: 2.1 time: 0.122907 Jul 20 15:25:11.522077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1803/2730] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:25:11 2022] DELETE /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:25:11.526987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:11.527697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:11.533329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.533769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.534509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.534931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.671770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:11.680127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-da8c49fe-f209-45e8-af64-638cb8a9ce5a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.156913 Jul 20 15:25:11.680700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1804/2731] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:11 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:11.685115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:11.685721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:11.691839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.692270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.693037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.693468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.839949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:11.850157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ae901bf5-61c2-47fc-9d9d-94d01605bc96 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1507 microversion: 2.1 time: 0.168496 Jul 20 15:25:11.850753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1805/2732] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:11 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1507 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:11.855126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:11.855724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:11.861873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.862357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:11.863140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:11.863581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.004366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.012481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dfe52979-ac6a-43f5-9841-803b14c6e22f tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.160708 Jul 20 15:25:12.013019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1806/2733] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:11 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:12.017561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:12.018159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:12.023972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.024411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.025155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.025599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.156141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.164634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.166123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.167263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0c77bb2c-c22c-4304-9f0c-8773cd2c3df8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.153267 Jul 20 15:25:12.167826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1807/2734] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:12 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:12.417949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:12.418817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:12.427721 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.428167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.428927 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.429382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.433430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:12.561767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.571811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-86b9e63d-5c48-42b0-944e-105f6cbbf4ba tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.160361 Jul 20 15:25:12.572512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1808/2735] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:12 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:12.698063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:12.698974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:12.706404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.706849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.707664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.708134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.846277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:12.854309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c353a1c2-4980-4f74-978a-18b7b281a4f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.161106 Jul 20 15:25:12.855083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1809/2736] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:12 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:12.869989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:12.870593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:12.877041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.877479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:12.878269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:12.878689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.018824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.026768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-113c99c0-1297-4f01-8a6f-b0ac7aa370c5 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1507 microversion: 2.1 time: 0.160520 Jul 20 15:25:13.027390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1810/2737] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:12 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1507 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:13.031817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:13.032415 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:13.038498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.038919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.039777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.040211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.223578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.242528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7552d7cc-fe4a-4cbd-b2a0-17e7319a8f22 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.214061 Jul 20 15:25:13.244636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1811/2738] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:13 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:13.251170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:13.252592 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:13.259519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.260213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.261178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.262148 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.501293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.509218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.510783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.511911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0d831fa9-245c-4648-87b7-150e7113aef1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.266073 Jul 20 15:25:13.512467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1812/2739] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:13 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:13.591704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:13.592691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:13.605686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.606436 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.607763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.608480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.766449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:13.775006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e9b7b78-7931-4ad0-971d-162db66555d6 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.189601 Jul 20 15:25:13.775700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1813/2740] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:13 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:13.876629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:13.877621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:13.887757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.888498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:13.889748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:13.890465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.085775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.093432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2a678ad2-9ee3-4bfe-ba3c-cc287f2d4a65 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.223121 Jul 20 15:25:14.093972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1814/2741] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:13 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:14.098399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:14.098990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:14.104957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.105403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.106157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.106598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.236561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.247013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1f1bd24-167e-482e-b5c3-66e21bf82fb6 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1507 microversion: 2.1 time: 0.151967 Jul 20 15:25:14.247785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1815/2742] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:14 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1507 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:14.259931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:14.260538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:14.266999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.267535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.268246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.268685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.407499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.415704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c0688644-fa8a-475a-83ef-97499a88a17a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.159375 Jul 20 15:25:14.416284 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1816/2743] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:14 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:14.530667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:14.531831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:14.537698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.538138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.538886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.539346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.670544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.678882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.680405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:14.681513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d009f16-0230-4f7e-b1a0-47c1a9ebdca5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.155239 Jul 20 15:25:14.682080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1817/2744] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:14 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:14.791575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:14.792179 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:14.798559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.798984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:14.799886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:14.800391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.081197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:15.089421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bacc0a3c-bff5-405d-88f3-088b52181fc7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1362 microversion: 2.1 time: 0.301727 Jul 20 15:25:15.093469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1818/2745] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:14 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1362 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:15.100254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-7abd8013-0fd5-4567-ada8-2839290e493c req-6b8fe53d-7970-4cc2-98cd-c74adfccdb56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a94cc383-ac2c-4750-96b4-93a6110f0203", "name": "network-vif-plugged", "status": "completed", "tag": "10fab4eb-f028-49c9-9cdf-35861d81feb7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:15.110766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7abd8013-0fd5-4567-ada8-2839290e493c req-6b8fe53d-7970-4cc2-98cd-c74adfccdb56 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.111762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7abd8013-0fd5-4567-ada8-2839290e493c req-6b8fe53d-7970-4cc2-98cd-c74adfccdb56 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.125266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-7abd8013-0fd5-4567-ada8-2839290e493c req-6b8fe53d-7970-4cc2-98cd-c74adfccdb56 service nova] Creating event network-vif-plugged:10fab4eb-f028-49c9-9cdf-35861d81feb7 for instance a94cc383-ac2c-4750-96b4-93a6110f0203 on hv-cinder-84902 Jul 20 15:25:15.130689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-7abd8013-0fd5-4567-ada8-2839290e493c req-6b8fe53d-7970-4cc2-98cd-c74adfccdb56 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034706 Jul 20 15:25:15.131646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1819/2746] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:15.136928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:15.137691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:15.144259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.145003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.145776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.146465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.368000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:15.375769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c7a49466-222a-4bb5-a9dd-fd2fee477b78 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.243000 Jul 20 15:25:15.376553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1820/2747] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:15 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:15.380885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:15.381489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:15.387353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.387809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.388570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.389006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.487101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:15.657331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:15.664966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fba99c27-93a1-4f0c-9fbc-bb75845480bf tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1507 microversion: 2.1 time: 0.287570 Jul 20 15:25:15.665717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1821/2748] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:15 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1507 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:15.670477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:15.670685 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:15.676544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.676996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.677755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.678189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.718618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=38 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:15.814234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:15.834337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-548668c7-3714-4d17-aff2-8b8e6d86d6a4 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.167649 Jul 20 15:25:15.835373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1822/2749] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:15 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:15.841506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-b9c30daa-400c-4823-8335-0d0d819f08e2 req-237d758f-4a46-4541-a157-03a396162d61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7d10b3f0-fe83-49d1-93c0-013f2e012d2d", "name": "network-vif-deleted", "tag": "1040fd9b-dd5d-4f60-9515-4b26b66dbcba"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:15.851027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b9c30daa-400c-4823-8335-0d0d819f08e2 req-237d758f-4a46-4541-a157-03a396162d61 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.851548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-b9c30daa-400c-4823-8335-0d0d819f08e2 req-237d758f-4a46-4541-a157-03a396162d61 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.866363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-b9c30daa-400c-4823-8335-0d0d819f08e2 req-237d758f-4a46-4541-a157-03a396162d61 service nova] Creating event network-vif-deleted:1040fd9b-dd5d-4f60-9515-4b26b66dbcba for instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d on hv-cinder-84902 Jul 20 15:25:15.878593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-b9c30daa-400c-4823-8335-0d0d819f08e2 req-237d758f-4a46-4541-a157-03a396162d61 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041612 Jul 20 15:25:15.880479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1823/2750] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:15.887744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:15.888740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:15.896959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.897948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:15.898929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:15.899722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.147951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.158561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.160155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.161252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f625d3df-5ac7-4cfc-ab67-6e7c8818944b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.278975 Jul 20 15:25:16.161816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1824/2751] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:15 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:16.166276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:16.166866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:16.174030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.174797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.175619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.176077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.372314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.380182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-64243636-8a9a-4060-ac85-6980222828de tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1548 microversion: 2.1 time: 0.217293 Jul 20 15:25:16.381117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1825/2752] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:16 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1548 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:16.392511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:16.393243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:16.399735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.400210 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.401024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.401523 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.405769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:16.530445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.538476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b370efe1-e423-4a22-a7fc-b6bbdeb00e10 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.149786 Jul 20 15:25:16.539033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1826/2753] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:16 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:25:16.543804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:16.544291 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:16.550131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.550564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.551343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.551777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.631716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-08d261c4-4e90-4722-8a85-245173257336 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.091127 Jul 20 15:25:16.632341 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1827/2754] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:25:16 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:16.681672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:16.682278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:16.688392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.688843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.689596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.690063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.721359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.721782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.722202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'fault' on Instance uuid 7d10b3f0-fe83-49d1-93c0-013f2e012d2d {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:25:16.781024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:16.790493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fe60bcba-8128-4a82-ba96-0db2aff03c10 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 200 len: 1312 microversion: 2.1 time: 0.112646 Jul 20 15:25:16.799964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1828/2755] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:16 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 1312 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:16.852078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:16.852692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:16.859188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.859639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:16.860411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:16.861378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.094941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:17.103138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-af5855d4-6c20-4059-991c-d5c4e55a14a6 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.255257 Jul 20 15:25:17.104088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1829/2756] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:16 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:25:17.147668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-dfb0da43-2dac-4216-8c8b-4ac91cf48655 req-04dd6fde-e6be-4a01-97c1-6177537ab138 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbbcd35f-cd1f-4195-958f-1e9d1194aea0", "name": "network-vif-plugged", "status": "completed", "tag": "c05d9926-8cc3-4a5e-8246-68dd7a0f2770"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:17.151437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:17.158887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-dfb0da43-2dac-4216-8c8b-4ac91cf48655 req-04dd6fde-e6be-4a01-97c1-6177537ab138 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.159471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-dfb0da43-2dac-4216-8c8b-4ac91cf48655 req-04dd6fde-e6be-4a01-97c1-6177537ab138 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.173203 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-dfb0da43-2dac-4216-8c8b-4ac91cf48655 req-04dd6fde-e6be-4a01-97c1-6177537ab138 service nova] Creating event network-vif-plugged:c05d9926-8cc3-4a5e-8246-68dd7a0f2770 for instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 on hv-cinder-84902 Jul 20 15:25:17.178398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-dfb0da43-2dac-4216-8c8b-4ac91cf48655 req-04dd6fde-e6be-4a01-97c1-6177537ab138 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034533 Jul 20 15:25:17.179011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1830/2757] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:17.184502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:17.185176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:17.191909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.192353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.193120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.193559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.572883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:17.581110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:17.582661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:17.583860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5904d9a7-aa6d-44ef-bba7-b3e38f6951b8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.403741 Jul 20 15:25:17.584418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1831/2758] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:17 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:17.588717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:17.589303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:17.595362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.595801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.596586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.597010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.856590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:17.864698 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebc53635-933d-4c5b-9abc-ac09b9ab8497 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.279285 Jul 20 15:25:17.866501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1832/2759] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:17 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:17.869710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:17.870326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 7d10b3f0-fe83-49d1-93c0-013f2e012d2d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:17.876295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.876758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.877513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.878056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.891481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d HTTP/1.1 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:25:17.893113 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. Jul 20 15:25:17.894797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '7d10b3f0-fe83-49d1-93c0-013f2e012d2d'} Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:25:17.895752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. Jul 20 15:25:17.897152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:17.898466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 404 to user: Instance 7d10b3f0-fe83-49d1-93c0-013f2e012d2d could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:25:17.898466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85c24117-2e61-4ee1-ab7b-1767351ed514 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d" status: 404 len: 111 microversion: 2.1 time: 0.028301 Jul 20 15:25:17.898466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1833/2760] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:17 2022] GET /compute/v2.1/servers/7d10b3f0-fe83-49d1-93c0-013f2e012d2d => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:25:17.903142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:17.903631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:17.909566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.909998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.910768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:17.911222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:17.936590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:25:18.017751 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c6372b9e-697c-4230-b530-7ffb7fe149b0 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "DELETE /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 204 len: 0 microversion: 2.1 time: 0.117786 Jul 20 15:25:18.018233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1834/2761] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:25:17 2022] DELETE /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:25:18.026863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:18.027677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:18.033521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.034131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.034892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.035354 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.218459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:18.226360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-774db709-7016-4d2d-b214-0b72397955c8 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.203721 Jul 20 15:25:18.226948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1835/2762] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:18 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:18.231454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:18.232070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:18.238267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.238712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.239508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.239939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.377819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:18.385731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a88f7c67-b17a-4f5b-875c-aac57ac279d9 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.157732 Jul 20 15:25:18.386272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1836/2763] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:18 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:18.390676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-efb682b9-0c4e-460b-91d2-b038f7103e4e req-d522fa38-ed45-435d-9fb2-6a59fa5321a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae", "tag": "b9ddd43f-d051-4dea-8402-f48e7e16421e"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:18.399026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-efb682b9-0c4e-460b-91d2-b038f7103e4e req-d522fa38-ed45-435d-9fb2-6a59fa5321a8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.399584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-efb682b9-0c4e-460b-91d2-b038f7103e4e req-d522fa38-ed45-435d-9fb2-6a59fa5321a8 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.412553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-efb682b9-0c4e-460b-91d2-b038f7103e4e req-d522fa38-ed45-435d-9fb2-6a59fa5321a8 service nova] Creating event network-changed:b9ddd43f-d051-4dea-8402-f48e7e16421e for instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae on hv-cinder-84902 Jul 20 15:25:18.417575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-efb682b9-0c4e-460b-91d2-b038f7103e4e req-d522fa38-ed45-435d-9fb2-6a59fa5321a8 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030154 Jul 20 15:25:18.418169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1837/2764] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:18.605289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:18.606031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:18.615892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.616512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.617604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.618241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.754007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:18.762035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:18.763656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:18.764884 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c5f72f39-3ce2-4c8b-90c6-741de3431239 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.166238 Jul 20 15:25:18.765412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1838/2765] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:18 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:18.882550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:18.883098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:18.889683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.890152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:18.890912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:18.891420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.032638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:19.040413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-73aca505-fc28-4e35-a5fe-ec306b50e9f8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.161802 Jul 20 15:25:19.040982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1839/2766] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:18 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:19.244052 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:19.248480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:19.259222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.259772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.260611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.261116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.413127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:19.420830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bba3cdcf-41db-48e7-a1a9-82092e8cf079 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.180899 Jul 20 15:25:19.421394 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1840/2767] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:19 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:25:19.425972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:19.426585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:19.432612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.433535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.434302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.434894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.572650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:19.581014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84547a42-e173-4519-b788-f43a50f8fd95 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1319 microversion: 2.1 time: 0.158610 Jul 20 15:25:19.581822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1841/2768] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:19 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:19.794230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:19.795500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:19.804451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.804930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.805693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:19.806128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:19.990474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:19.998992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:20.000528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:20.002219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bb952a0b-1b12-484d-90ce-7136706727e1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.221872 Jul 20 15:25:20.003216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1842/2769] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:19 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:20.057360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:20.057980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:20.064879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.065376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.066137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.066576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.070707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:20.198259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:20.206078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1bcb04af-588f-4e28-b206-cf036b13c1fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.152409 Jul 20 15:25:20.207022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1843/2770] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:20 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:20.441862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:20.442846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:20.453229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.453981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.455276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.456040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.595659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:20.603392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1818183c-f504-44bc-bebb-47fa315438a9 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1522 microversion: 2.1 time: 0.168054 Jul 20 15:25:20.603955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1844/2771] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:20 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:20.608265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:20.608865 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:20.615584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.616023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.616804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.617242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.748032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:20.755939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f3c1b7c7-74cc-40d8-83a8-effe0b24f7b3 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1508 microversion: 2.1 time: 0.151040 Jul 20 15:25:20.756512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1845/2772] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:20 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:20.771788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "56e7ed8c-60eb-4564-bae1-2d93db837744", "device": "/dev/sdb"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:20.772946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:20.779176 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.779619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:20.780375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:20.780803 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:21.750259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c97d38e0-0c76-415c-9846-36adc55f6e2d tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "POST /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.981654 Jul 20 15:25:21.751092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1846/2773] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:25:20 2022] POST /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203/os-volume_attachments => generated 194 bytes in 983 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:21.758070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:21.759007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:21.765427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:21.765873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:21.766629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:21.767083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:21.984221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:21.992942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:21.994424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:21.995563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4375c21c-1c41-4b30-a95c-c70a48d73970 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.243539 Jul 20 15:25:21.996112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1847/2774] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:21 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:22.001553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:22.002286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:22.007975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:22.008423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:22.009174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:22.009604 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:22.254693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:22.263622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-68f15a5e-792c-4d6c-96f2-2ad4a5c455e7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1365 microversion: 2.1 time: 0.265445 Jul 20 15:25:22.264213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1848/2775] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:21 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1365 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:22.268790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:22.269404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:22.276556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:22.277102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:22.278041 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:22.278545 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:22.363447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:22.370897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-87bbda48-6ba3-4358-a770-07a17f1c6ce7 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1480 microversion: 2.1 time: 0.105716 Jul 20 15:25:22.371493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1849/2776] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:22 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1480 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:22.376050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-5d48726c-f4f9-41bc-b6e3-8d7ce7ec0bf5 req-2f4f5aa4-4355-42b6-9a80-1069ace911b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "304148dc-eb2a-4dec-a9ec-1cff5c5114b7", "name": "network-vif-deleted", "tag": "c215e006-b33e-46f8-be1f-7cc86ace4f37"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:22.384340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-5d48726c-f4f9-41bc-b6e3-8d7ce7ec0bf5 req-2f4f5aa4-4355-42b6-9a80-1069ace911b6 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:22.384851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-5d48726c-f4f9-41bc-b6e3-8d7ce7ec0bf5 req-2f4f5aa4-4355-42b6-9a80-1069ace911b6 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:22.396057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-5d48726c-f4f9-41bc-b6e3-8d7ce7ec0bf5 req-2f4f5aa4-4355-42b6-9a80-1069ace911b6 service nova] Creating event network-vif-deleted:c215e006-b33e-46f8-be1f-7cc86ace4f37 for instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 on hv-cinder-84902 Jul 20 15:25:22.401676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-5d48726c-f4f9-41bc-b6e3-8d7ce7ec0bf5 req-2f4f5aa4-4355-42b6-9a80-1069ace911b6 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029084 Jul 20 15:25:22.402371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1850/2777] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:23.014347 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:23.015978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:23.034004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.034722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.035785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.041881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.186001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:23.194117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:23.195642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:23.196742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3dfa9638-7cb7-40c9-8b05-a0064d0c5fbc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.186683 Jul 20 15:25:23.197295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1851/2778] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:23 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:25:23.202280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-2c3f3b87-60d0-4e03-a9fd-e331e885ee6f req-7aa5dea3-25be-4d22-81a6-78568e22e367 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "name": "network-vif-plugged", "status": "completed", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:23.210606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2c3f3b87-60d0-4e03-a9fd-e331e885ee6f req-7aa5dea3-25be-4d22-81a6-78568e22e367 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.211032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2c3f3b87-60d0-4e03-a9fd-e331e885ee6f req-7aa5dea3-25be-4d22-81a6-78568e22e367 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.222022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-2c3f3b87-60d0-4e03-a9fd-e331e885ee6f req-7aa5dea3-25be-4d22-81a6-78568e22e367 service nova] Creating event network-vif-plugged:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:25:23.227199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-2c3f3b87-60d0-4e03-a9fd-e331e885ee6f req-7aa5dea3-25be-4d22-81a6-78568e22e367 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028098 Jul 20 15:25:23.227796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1852/2779] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:23.281609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:23.282333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:23.289430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.290001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.290795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.291271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.433156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:23.442270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ca0271f3-5070-4782-8ddd-9a9e86ad7be8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1553 microversion: 2.1 time: 0.164551 Jul 20 15:25:23.443072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1853/2780] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:23 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1553 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:23.447457 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:23.448104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:23.454126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.454588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.455357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.455855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.487869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.488395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.488848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lazy-loading 'fault' on Instance uuid 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:25:23.530737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '938083ff74d84224be142a26d8e17ad6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5c6ce84a298f4daa938864ec78d7a996', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:23.538554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8a631c0a-679f-431c-9676-d97eba620429 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 200 len: 1312 microversion: 2.1 time: 0.094548 Jul 20 15:25:23.539129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1854/2781] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:23 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 1312 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:23.543877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:23.544351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:23.552963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.554240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.555220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:23.555654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:23.642317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5032101a-1cf9-4f8a-8277-8a15692e5853 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.102122 Jul 20 15:25:23.643128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1855/2782] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:25:23 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-interface => generated 284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:24.214797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:24.215584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:24.222051 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:24.222577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:24.223470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:24.223899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:24.357957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:24.366550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:24.368121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:24.369227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c0892062-0c48-4ecc-ae19-7ed6e4b071ce tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1407 microversion: 2.20 time: 0.159300 Jul 20 15:25:24.369790 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1856/2783] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:24 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1407 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:24.555764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:24.556468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] [instance: 304148dc-eb2a-4dec-a9ec-1cff5c5114b7] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:24.566115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:24.567796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:24.567796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:24.568766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:24.583248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] HTTP exception thrown: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Request method failure captured: Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 HTTP/1.1 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWlc2FycCzH5xiJC4xExZYmgffLIsqiY9S5L4DwEIH5slPZq9KH43vGiYQYJ1gCevFSsGRHKWqL6M5IZcdTbW7C0gS1C0FqF6ZwBWtuXSTRxJZ1HMNm5T9tmKYnCJtf1qFTz0xC8_-4sGxr3aSLkpXykxs5YTD8OHma62v4eijhHto Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:25:24.584983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5c6ce84a298f4daa938864ec78d7a996"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5c6ce84a298f4daa938864ec78d7a996"}]}] Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5c6ce84a298f4daa938864ec78d7a996 Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1168846897 Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 938083ff74d84224be142a26d8e17ad6 Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1168846897-project Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. Jul 20 15:25:24.586762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '304148dc-eb2a-4dec-a9ec-1cff5c5114b7'} Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:25:24.588682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. Jul 20 15:25:24.590279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:24.592368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] Returning 404 to user: Instance 304148dc-eb2a-4dec-a9ec-1cff5c5114b7 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:25:24.592368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5ca5eb1a-4dbf-4174-8020-089af123e2f3 tempest-ServerRescueNegativeTestJSON-1168846897 tempest-ServerRescueNegativeTestJSON-1168846897-project] 10.222.0.15 "GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7" status: 404 len: 111 microversion: 2.1 time: 0.034917 Jul 20 15:25:24.592368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1857/2784] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:24 2022] GET /compute/v2.1/servers/304148dc-eb2a-4dec-a9ec-1cff5c5114b7 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:25:24.614888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-3915baa1-2a2f-45a9-adb0-20a1fb52c3a8 req-d7c6660c-d709-4eb6-b61d-205c710bad35 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbbcd35f-cd1f-4195-958f-1e9d1194aea0", "tag": "c05d9926-8cc3-4a5e-8246-68dd7a0f2770"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:24.623538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3915baa1-2a2f-45a9-adb0-20a1fb52c3a8 req-d7c6660c-d709-4eb6-b61d-205c710bad35 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:24.623990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3915baa1-2a2f-45a9-adb0-20a1fb52c3a8 req-d7c6660c-d709-4eb6-b61d-205c710bad35 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:24.638615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-3915baa1-2a2f-45a9-adb0-20a1fb52c3a8 req-d7c6660c-d709-4eb6-b61d-205c710bad35 service nova] Creating event network-changed:c05d9926-8cc3-4a5e-8246-68dd7a0f2770 for instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 on hv-cinder-84902 Jul 20 15:25:24.643802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-3915baa1-2a2f-45a9-adb0-20a1fb52c3a8 req-d7c6660c-d709-4eb6-b61d-205c710bad35 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032834 Jul 20 15:25:24.644661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1858/2785] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:25.392380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:25.394361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:25.404683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:25.405136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:25.406391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:25.407130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:25.565938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:25.574638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:25.576157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:25.577327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3270921a-766c-4d93-b97e-e6f5dc28b7b9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1600 microversion: 2.20 time: 0.192795 Jul 20 15:25:25.578060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1859/2786] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:25:25 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1600 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:25:25.596363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:25.596862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:25.604650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:25.604650 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:25.605282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:25.605719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:25.703801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-067d55fe-2e0b-4dfd-b5c4-ec887d50e231 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.111405 Jul 20 15:25:25.704982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1860/2787] 10.222.0.15 () {60 vars in 1315 bytes} [Wed Jul 20 15:25:25 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-interface => generated 285 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:25:27.167961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-1df108ef-c1b6-46f0-9fde-78fb09fef0c0 req-5a1a5470-3dc8-48fb-baf8-4490b244d244 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:27.177940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-1df108ef-c1b6-46f0-9fde-78fb09fef0c0 req-5a1a5470-3dc8-48fb-baf8-4490b244d244 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:27.178534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-1df108ef-c1b6-46f0-9fde-78fb09fef0c0 req-5a1a5470-3dc8-48fb-baf8-4490b244d244 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:27.197156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-1df108ef-c1b6-46f0-9fde-78fb09fef0c0 req-5a1a5470-3dc8-48fb-baf8-4490b244d244 service nova] Creating event network-changed:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:25:27.204077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-1df108ef-c1b6-46f0-9fde-78fb09fef0c0 req-5a1a5470-3dc8-48fb-baf8-4490b244d244 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039819 Jul 20 15:25:27.205687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1861/2788] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:25:47.490154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:47.490799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:47.493892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:47.494530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:25:47.500226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:47.500661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:47.501413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:47.501881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:47.586792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lazy-loading 'services' on Instance uuid a94cc383-ac2c-4750-96b4-93a6110f0203 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:25:47.670767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-65d26672-4bf5-413c-9054-be21b347cd20 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203/os-volume_attachments/56e7ed8c-60eb-4564-bae1-2d93db837744" status: 202 len: 0 microversion: 2.1 time: 0.184961 Jul 20 15:25:47.671437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1862/2789] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:25:47 2022] DELETE /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203/os-volume_attachments/56e7ed8c-60eb-4564-bae1-2d93db837744 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:25:48.797023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:48.797708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:48.814880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:48.818749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:48.821057 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:48.822175 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:48.858260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:25:48.950454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45b322c3-2772-4f8c-af40-8a0a3b386a84 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "DELETE /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 204 len: 0 microversion: 2.1 time: 0.157990 Jul 20 15:25:48.951158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1863/2790] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:25:48 2022] DELETE /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:25:48.960225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:48.961012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:48.967017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:48.967567 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:48.968358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:48.968852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:49.107110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:49.116627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-00c8da3e-ee78-44e7-9dce-89a74f0ffe98 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1514 microversion: 2.1 time: 0.160679 Jul 20 15:25:49.117202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1864/2791] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:48 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1514 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:50.137757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:50.138892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:50.149955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:50.150707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:50.152019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:50.152738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:50.287550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:50.297222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f2dd4341-2c14-446c-bf79-dc0a849f135a tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1514 microversion: 2.1 time: 0.165625 Jul 20 15:25:50.298233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1865/2792] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:50 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1514 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:51.320756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:51.321862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:51.331765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:51.332181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:51.332938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:51.333393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:51.465506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:51.474581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d08ed2ce-6d74-4fc5-ab36-781c4563bfc5 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1514 microversion: 2.1 time: 0.160406 Jul 20 15:25:51.475163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1866/2793] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:51 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1514 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:52.424050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-14d6519d-7928-47c0-9deb-8f1dc1f40ad3 req-03ad2822-36a9-4ca3-969e-049b8c172ee3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a94cc383-ac2c-4750-96b4-93a6110f0203", "name": "network-vif-deleted", "tag": "10fab4eb-f028-49c9-9cdf-35861d81feb7"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:25:52.436105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-14d6519d-7928-47c0-9deb-8f1dc1f40ad3 req-03ad2822-36a9-4ca3-969e-049b8c172ee3 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:52.437556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-14d6519d-7928-47c0-9deb-8f1dc1f40ad3 req-03ad2822-36a9-4ca3-969e-049b8c172ee3 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:52.449942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-14d6519d-7928-47c0-9deb-8f1dc1f40ad3 req-03ad2822-36a9-4ca3-969e-049b8c172ee3 service nova] Creating event network-vif-deleted:10fab4eb-f028-49c9-9cdf-35861d81feb7 for instance a94cc383-ac2c-4750-96b4-93a6110f0203 on hv-cinder-84902 Jul 20 15:25:52.454963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-14d6519d-7928-47c0-9deb-8f1dc1f40ad3 req-03ad2822-36a9-4ca3-969e-049b8c172ee3 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034741 Jul 20 15:25:52.455619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1867/2794] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:25:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:25:52.492397 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:52.495339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:52.508261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:52.509011 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:52.510119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:52.510390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:52.593472 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bc2f1a0d431341d6a5d1a8c5de131ece', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1cdab962e8b0462b8d1d28dcde703c15', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:25:52.603617 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-29c8a5e3-e363-48d4-9515-229b46578167 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 200 len: 1304 microversion: 2.1 time: 0.115491 Jul 20 15:25:52.604285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1868/2795] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:52 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 1304 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:25:53.620102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:25:53.620741 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] [instance: a94cc383-ac2c-4750-96b4-93a6110f0203] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:25:53.627957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:53.628410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:53.629160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:25:53.629593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:25:53.644705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] HTTP exception thrown: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Request method failure captured: Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 HTTP/1.1 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2Ba2Da-ehBHDesIrcgDFA3lA1KgQrWEUFDIeCHqgOF9KhPcNdR6P2u1G8xQyRZGgAgjeXMgX_c5FhDetQdAHZ8Bt6WCqSXoS2rFn_mggGc7thSZvMlISdU0eg7gYUW8SiIF4C3j_SUhctUlKAg__5eNzle3kG2QmJW6mEl_ZctiuMq0 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:25:53.646366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/1cdab962e8b0462b8d1d28dcde703c15"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/1cdab962e8b0462b8d1d28dcde703c15"}]}] Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 1cdab962e8b0462b8d1d28dcde703c15 Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1635253899 Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: bc2f1a0d431341d6a5d1a8c5de131ece Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesSnapshotTestJSON-1635253899-project Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. Jul 20 15:25:53.648386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'a94cc383-ac2c-4750-96b4-93a6110f0203'} Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:25:53.649459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. Jul 20 15:25:53.651245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:25:53.652697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] Returning 404 to user: Instance a94cc383-ac2c-4750-96b4-93a6110f0203 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:25:53.652697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3c353fcc-1bed-479a-91f7-e62b84911840 tempest-VolumesSnapshotTestJSON-1635253899 tempest-VolumesSnapshotTestJSON-1635253899-project] 10.222.0.15 "GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203" status: 404 len: 111 microversion: 2.1 time: 0.031579 Jul 20 15:25:53.652697 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1869/2796] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:25:53 2022] GET /compute/v2.1/servers/a94cc383-ac2c-4750-96b4-93a6110f0203 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:26:24.521267 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:26:24.521803 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:26:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:26:28.270573 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:26:28.271273 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:26:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:26:48.818558 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:26:48.819472 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:26:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:26:49.161205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a63fb03f-2c8c-4558-8325-bb4657dc0010 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'create', calling method: .version_select at 0x7f7b15b01550>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-653951357", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEFtC0Rj0A0HYTWTNc/1fmSO56PXfy5YM4rpRRnezG9byyNhnbPJ3Wo0GA4uOoWzNxh0iUv8Pl8s7GukqS/e/qA9R/YRAVooGaa13pKLO6WA/lI1IG07A3BkFPsFTEcqIA=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:26:49.165605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.166252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=51,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:26:49.180107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-a63fb03f-2c8c-4558-8325-bb4657dc0010 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Getting quotas for project 535d117f782341d5aa579a8fb34c35c5. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:26:49.184413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-a63fb03f-2c8c-4558-8325-bb4657dc0010 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Getting quotas for user 9b70cbf2ad4f405b84cbea240b83cd05 and project 535d117f782341d5aa579a8fb34c35c5. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:26:49.207595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a63fb03f-2c8c-4558-8325-bb4657dc0010 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.050317 Jul 20 15:26:49.208227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1870/2797] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 15:26:49 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:26:49.262183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-475721180", "imageRef": "a952ad58-7265-478b-8d51-6497823f180f", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-653951357", "networks": [{"uuid": "f08c9c47-7bf9-4a23-93fa-a2fd4c1b4f15"}]}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:26:49.433447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] validate_networks() for [('f08c9c47-7bf9-4a23-93fa-a2fd4c1b4f15', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:26:49.846755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:26:49.847443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:26:49.848193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:26:49.848849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:26:49.849412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:26:49.862716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:49.863208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:49.864102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:49.864529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:49.882289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Getting quotas for project 535d117f782341d5aa579a8fb34c35c5. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:26:49.887095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Getting quotas for user 9b70cbf2ad4f405b84cbea240b83cd05 and project 535d117f782341d5aa579a8fb34c35c5. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:26:49.899428 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:26:49.901805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a952ad58-7265-478b-8d51-6497823f180f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:26:49.955968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d75fc346-5d19-41cd-b605-66bafa1dcca6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.697746 Jul 20 15:26:49.956515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1871/2798] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 15:26:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 699 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 15:26:49.966456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0441f49-6d71-4048-be40-14a94326e8e6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:49.967076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0441f49-6d71-4048-be40-14a94326e8e6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:50.023833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e0441f49-6d71-4048-be40-14a94326e8e6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:50.031087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0441f49-6d71-4048-be40-14a94326e8e6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1257 microversion: 2.1 time: 0.068346 Jul 20 15:26:50.031786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1872/2799] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:49 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1257 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:26:51.049264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:51.050020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:51.058350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:51.058804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:51.059609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:51.060058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:51.161797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:51.169246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-04c7345c-a938-40e1-9537-6012b89db27f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1317 microversion: 2.1 time: 0.124463 Jul 20 15:26:51.170060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1873/2800] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:51 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1317 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:26:52.186657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:52.187368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:52.194318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:52.194736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:52.195776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:52.196214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:52.338172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:52.346294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab6ae538-51c4-48ee-9d41-033e6cef3bee tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.163615 Jul 20 15:26:52.346895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1874/2801] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:52 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:53.364064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:53.364813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:53.372419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:53.372932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:53.373702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:53.374139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:53.515694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:53.523686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-369f7fb6-a7da-404e-a0bd-c8ea6627ecf6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.163441 Jul 20 15:26:53.524339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1875/2802] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:53 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:54.539863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:54.540571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:54.547706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:54.548158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:54.548921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:54.549361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:54.683001 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:54.691867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-051c73d7-7949-4284-8426-4ca6bba6466d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.155850 Jul 20 15:26:54.692531 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1876/2803] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:54 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:55.709539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:55.710400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:55.717189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:55.717631 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:55.718386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:55.718813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:55.857488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:55.865574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72020e50-7738-462d-9d88-d23f1e46b3bd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.159976 Jul 20 15:26:55.866131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1877/2804] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:55 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:56.882349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:56.883097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:56.889828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:56.890276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:56.891003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:56.891484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:57.022614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:57.031343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f5db650-88a1-445e-aaa3-a6c8d5f490d1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.152664 Jul 20 15:26:57.032302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1878/2805] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:56 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:58.048966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:58.049766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:58.056476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:58.056926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:58.057682 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:58.058107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:58.198459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:58.206271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce2e15c2-1c60-4981-b260-9590e514c6c2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.161320 Jul 20 15:26:58.206832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1879/2806] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:58 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:26:59.223690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:26:59.224407 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:26:59.231798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:59.232255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:59.233249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:26:59.234030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:26:59.366125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:26:59.373871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-41b08dcd-0d7f-4694-a882-066927729a1a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.154066 Jul 20 15:26:59.374389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1880/2807] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:26:59 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:00.391680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:00.392443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:00.394285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:00.394983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=12 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:00.401569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:00.401949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:00.402716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:00.403346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:00.542479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:00.550255 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b8b832fe-46a0-4bb9-94f1-18efc28a141a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.162982 Jul 20 15:27:00.550895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1881/2808] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:00 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:01.568619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:01.569482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:01.579618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:01.580075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:01.580847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:01.581282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:01.715467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:01.723345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d35239ca-993f-4b2a-a0cd-913bfd7a22b6 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.158712 Jul 20 15:27:01.723955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1882/2809] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:01 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:02.742161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:02.742922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:02.750288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:02.750823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:02.751732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:02.752244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:02.908613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:02.916813 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1e7fa62b-f80e-49a7-96fa-967532de8352 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.179693 Jul 20 15:27:02.917636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1883/2810] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:02 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:03.935103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:03.935915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:03.943263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:03.943952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:03.945162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:03.945846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:04.081087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:04.088972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eea36bbf-fc36-449e-bc20-8f487e87e798 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.157937 Jul 20 15:27:04.089544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1884/2811] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:03 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:05.109572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:05.110468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:05.120634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:05.121326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:05.122479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:05.123162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:05.254307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:05.261874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e393cc7f-2d46-4994-a4e9-c905ff2e8e93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.158971 Jul 20 15:27:05.262441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1885/2812] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:05 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:06.283950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:06.285090 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:06.296046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:06.296824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:06.297576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:06.297999 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:06.440992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:06.449009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2eb0eda3-c7f1-41d3-b544-d4d75f3f6a04 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.172441 Jul 20 15:27:06.449576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1886/2813] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:06 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:07.475925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:07.476855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:07.485978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:07.486613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:07.487729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:07.488353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:07.626387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:07.634170 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba115fd6-b0d5-4295-abd2-22f5aa9a902c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.164376 Jul 20 15:27:07.634776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1887/2814] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:07 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:08.655532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:08.656556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:08.667570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:08.668316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:08.669560 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:08.670296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:08.799163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:08.807356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-563dd3e8-b489-47df-8d8b-6a69fd01573a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.158329 Jul 20 15:27:08.807944 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1888/2815] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:08 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:09.824816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:09.825536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:09.832445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:09.832883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:09.833647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:09.834084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:09.838184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:09.965547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:09.973297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-19ad4879-9ee9-4bd0-ac3b-f915d27731dd tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.152414 Jul 20 15:27:09.973852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1889/2816] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:09 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:10.994368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:10.996209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:11.004667 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:11.005120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:11.005901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:11.006333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:11.141968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:11.153480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba9b0115-ca58-4975-9f72-002f10346f4a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.165393 Jul 20 15:27:11.154280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1890/2817] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:10 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:12.174579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:12.175516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:12.183396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:12.183928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:12.184829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:12.185336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:12.324688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:12.333416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-889209f8-cb64-4d21-b384-68f5c26cce9d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.164716 Jul 20 15:27:12.333955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1891/2818] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:12 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:13.353848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:13.354946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:13.366188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:13.366928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:13.368222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:13.368947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:13.504102 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:13.511836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8caa08ed-78f3-43d6-9da0-6ce8e453e66e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.164567 Jul 20 15:27:13.512400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1892/2819] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:13 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:14.533318 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:14.534933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:14.542033 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:14.543827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:14.544606 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:14.545034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:14.857535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:14.865957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bdba567-8982-4ad8-aa4d-2fe219d74eb8 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.339741 Jul 20 15:27:14.866548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1893/2820] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:14 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:15.060024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-2ed22b8a-604a-4405-a5a1-b75a43adde5b req-202bff99-0f89-4d8a-9402-816b8644b489 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:15.068917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2ed22b8a-604a-4405-a5a1-b75a43adde5b req-202bff99-0f89-4d8a-9402-816b8644b489 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:15.069419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-2ed22b8a-604a-4405-a5a1-b75a43adde5b req-202bff99-0f89-4d8a-9402-816b8644b489 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:15.239528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-2ed22b8a-604a-4405-a5a1-b75a43adde5b req-202bff99-0f89-4d8a-9402-816b8644b489 service nova] Creating event network-changed:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:27:15.244468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-2ed22b8a-604a-4405-a5a1-b75a43adde5b req-202bff99-0f89-4d8a-9402-816b8644b489 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.294147 Jul 20 15:27:15.245093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1894/2821] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:15.887950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:15.889097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:15.896739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:15.897178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:15.897917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:15.898336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:16.033359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:16.041363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dbc883cc-0177-4831-a7bf-63ecf664a9e1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.160383 Jul 20 15:27:16.041958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1895/2822] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:15 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:17.063021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:17.063883 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:17.071556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:17.072087 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:17.072971 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:17.073482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:17.215581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:17.231010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7693a85f-8428-4f26-af4b-c9a4848ebd68 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.174174 Jul 20 15:27:17.231628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1896/2823] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:17 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:18.248995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:18.249953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:18.261847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:18.262294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:18.263083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:18.263539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:18.401562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:18.413783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4e30ab5b-337a-40a7-b511-10006c9dbd93 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.168896 Jul 20 15:27:18.414705 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1897/2824] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:18 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:19.435288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:19.436361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:19.439086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=4 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:19.445092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:19.445519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:19.446271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:19.446723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:19.579420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:19.588065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a44ac7f3-2bea-41e7-968d-31eca4f959a0 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.159280 Jul 20 15:27:19.588639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1898/2825] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:19 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:20.145264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:20.145757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:20.152287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:20.152732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:20.153481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:20.153926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:20.177542 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c38eb2c6-b0f7-426e-9c44-863c3d012d21 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/ips" status: 200 len: 146 microversion: 2.1 time: 0.159882 Jul 20 15:27:20.178096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1899/2826] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 15:27:20 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/ips => generated 146 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:20.604847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:20.605524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:20.612765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:20.613220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:20.613985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:20.614418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:20.762035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:20.773503 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7f1dd80a-f06c-4b43-b4a2-88af40b6ba21 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.172585 Jul 20 15:27:20.774269 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1900/2827] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:20 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:21.511334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0ace56e1-fae7-49c2-82a7-0eaa38b02afb"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:21.512508 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:21.519028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:21.519528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:21.520319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:21.521167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:21.800302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-00cbcc2d-2095-4951-aa28-b47c68e30d15 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.292485 Jul 20 15:27:21.801018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1901/2828] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:27:21 2022] POST /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 194 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:21.805696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-3346febf-1f26-401d-b530-524937f95e8a req-cfb34dc5-c9ea-4e1e-aa4e-2329d3903878 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "name": "network-vif-plugged", "status": "completed", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:21.814290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3346febf-1f26-401d-b530-524937f95e8a req-cfb34dc5-c9ea-4e1e-aa4e-2329d3903878 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:21.814727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3346febf-1f26-401d-b530-524937f95e8a req-cfb34dc5-c9ea-4e1e-aa4e-2329d3903878 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:21.826461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-3346febf-1f26-401d-b530-524937f95e8a req-cfb34dc5-c9ea-4e1e-aa4e-2329d3903878 service nova] Creating event network-vif-plugged:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:27:21.831496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-3346febf-1f26-401d-b530-524937f95e8a req-cfb34dc5-c9ea-4e1e-aa4e-2329d3903878 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029325 Jul 20 15:27:21.832304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1902/2829] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:21.836941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:21.837621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:21.843906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:21.844224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:21.845205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:21.845628 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:21.985236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:21.992844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f6e085a4-a5c4-4765-b9cd-75da81ef611c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.159622 Jul 20 15:27:21.993413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1903/2830] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:21 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:23.014226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:23.015303 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:23.026307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:23.027117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:23.028458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:23.029190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:23.174797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:23.186482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2d89681c-32b2-47d4-a96c-6286411739be tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.178463 Jul 20 15:27:23.187120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1904/2831] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:23 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:24.208485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:24.209498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:24.220286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:24.221042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:24.222307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:24.223032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:24.368168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:24.380338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-53b8c806-5804-4f92-9735-8fba558d5286 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1357 microversion: 2.1 time: 0.178195 Jul 20 15:27:24.381110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1905/2832] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:24 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1357 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:25.401924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:25.403015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:25.410949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.411455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.412195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.412633 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.416742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:25.553933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:25.562389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-89a4bd21-3edd-424d-a4e5-5675af9c702b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1546 microversion: 2.1 time: 0.167032 Jul 20 15:27:25.562968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1906/2833] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:25 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1546 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:25.578281 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:25.578921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:25.585036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.585486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.586242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.586673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.718248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:25.729885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3d1e3461-b126-430a-b1ef-851bda744697 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1546 microversion: 2.1 time: 0.154925 Jul 20 15:27:25.730679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1907/2834] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:25 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1546 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:25.747262 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:25.750603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:25.751395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Searching by: {'deleted': False, 'project_id': '535d117f782341d5aa579a8fb34c35c5'} {{(pid=124396) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 20 15:27:25.758222 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.758666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.759495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.759932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.851212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.multi_cell_list [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=124396) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 20 15:27:25.941906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-de062888-a7c6-4673-856b-c3bf09ff013c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.198474 Jul 20 15:27:25.942401 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1908/2835] 10.222.0.15 () {58 vars in 1124 bytes} [Wed Jul 20 15:27:25 2022] GET /compute/v2.1/servers => generated 342 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:25.952404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:25.953007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:25.959006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.959491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:25.960242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:25.960676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:26.100140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:26.108271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9177a207-66b3-4596-b20f-5b4cb37e6b9c tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1546 microversion: 2.1 time: 0.159166 Jul 20 15:27:26.108847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1909/2836] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:25 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1546 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:26.595651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:26.596368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:26.603551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:26.604044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:26.604860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:26.605332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:26.630130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ddb7ebd-12cf-4eb3-a46b-f5271036a221 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/ips" status: 200 len: 148 microversion: 2.1 time: 0.038742 Jul 20 15:27:26.630653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1910/2837] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 15:27:26 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/ips => generated 148 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:27.649897 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a81836ca-bd09-4ad6-8476-3e383ed3eea6"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:27.650945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:27.658187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:27.658651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:27.659434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:27.659862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:27.965227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e01d7fea-3a1e-45e2-999b-713b8964ce0f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.319428 Jul 20 15:27:27.965839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1911/2838] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:27:27 2022] POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments => generated 194 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:29.029026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "df4ce82f-a40d-44d4-b6af-47186a32f8f8"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:29.031532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:29.038276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:29.038720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:29.039529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:29.039965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:29.702108 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-80c4ed14-2780-4094-aaa7-6eca4942c47b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.678324 Jul 20 15:27:29.702726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1912/2839] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:27:29 2022] POST /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 194 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:29.823279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:29.823862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:29.830290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:29.830728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:29.831514 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:29.831946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:29.855493 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8c4ff342-da8f-4181-9bb4-834ff952bea3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/ips" status: 200 len: 153 microversion: 2.20 time: 0.151536 Jul 20 15:27:29.856043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1913/2840] 10.222.0.15 () {60 vars in 1288 bytes} [Wed Jul 20 15:27:29 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/ips => generated 153 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:27:30.391332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:30.392308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:30.398914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:30.399395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:30.400149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:30.400588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:30.432538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-092a4433-2370-4c55-94a5-8e6b84d28f8f tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045145 Jul 20 15:27:30.433091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1914/2841] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:30 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:30.442945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:30.443463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:30.449424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:30.449877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:30.450629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:30.451080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:30.479566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-302826bb-e258-49cd-9422-142a9ab268b4 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb" status: 200 len: 194 microversion: 2.1 time: 0.040105 Jul 20 15:27:30.482954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1915/2842] 10.222.0.15 () {58 vars in 1412 bytes} [Wed Jul 20 15:27:30 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb => generated 194 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:31.823286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:31.823922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:31.830915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:31.831386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:31.832142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:31.832572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:31.857183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lazy-loading 'flavor' on Instance uuid 70521d7d-d55b-40ff-a8b8-8579dcba3c98 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:27:32.121317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c22d439-98b8-4986-87f9-86a5ca76e6c3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/action" status: 202 len: 0 microversion: 2.20 time: 0.301877 Jul 20 15:27:32.121932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1916/2843] 10.222.0.15 () {62 vars in 1318 bytes} [Wed Jul 20 15:27:31 2022] POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/action => generated 0 bytes in 303 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:27:32.130806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:32.131500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:32.137807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:32.138220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:32.138994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:32.139473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:32.278960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:32.288603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:32.290208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:32.291485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-42ad64ac-d7b7-4139-ab93-e58a33e89aaf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.164621 Jul 20 15:27:32.292187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1917/2844] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:32 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:32.854236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cb10eea1-ff88-47da-af68-7a13027e52a5"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:32.855391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:32.862439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:32.863119 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:32.863642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:32.864076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:32.882007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:33.158827 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bbf76a54-ee89-427b-bfdd-78b370ee8e99 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "POST /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.308345 Jul 20 15:27:33.159574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1918/2845] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:27:32 2022] POST /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 194 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:33.314506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:33.315476 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:33.324023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:33.324648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:33.325644 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:33.326221 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:33.477975 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:33.489373 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:33.491874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:33.493688 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2b47b495-4115-4408-80dc-9f6e3bf00e21 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.185417 Jul 20 15:27:33.494539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1919/2846] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:33 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:34.517873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:34.518632 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:34.529335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:34.531304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:34.532083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:34.532517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:34.668226 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:34.676842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:34.678317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:34.679509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1abd9d6a-f90f-4eec-86bd-7379cc7fdc89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.167860 Jul 20 15:27:34.680072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1920/2847] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:34 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:35.701265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:35.702326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:35.713225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:35.714013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:35.715301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:35.716015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:35.791626 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:35.860716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:35.869026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:35.870501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:35.871702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7fcf77df-c376-4a54-841e-fbe839f0d3b6 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.176854 Jul 20 15:27:35.872261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1921/2848] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:35 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:36.893997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:36.895104 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:36.906130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:36.906864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:36.908209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:36.908908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:37.055180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:37.067581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:37.069805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:37.071588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfe28f2b-0530-4928-b90f-0cd5efcdf825 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.183985 Jul 20 15:27:37.072319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1922/2849] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:36 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:38.094613 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:38.095759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:38.106712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:38.107860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:38.109106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:38.109834 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:38.254879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:38.268596 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:38.271101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:38.273042 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-74bd0563-363a-4a0c-8a75-63dabe353359 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.184765 Jul 20 15:27:38.273960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1923/2850] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:38 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:38.823950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:38.824558 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:38.831075 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:38.831570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:38.832316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:38.832748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:38.970435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:38.979081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-36462cd8-1605-436d-a246-08c50e043dd9 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1592 microversion: 2.1 time: 0.158944 Jul 20 15:27:38.979676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1924/2851] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:38 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1592 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:39.292023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:39.292733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:39.299414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:39.299862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:39.300623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:39.301056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:39.513374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:39.521622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:39.523172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:39.524339 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a85c9507-0572-4893-b076-1508eb6aeeb1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.236269 Jul 20 15:27:39.524903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1925/2852] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:39 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:27:40.543364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:40.544771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:40.552513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:40.552976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:40.553748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:40.554181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:40.757018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:40.765211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:40.766743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:40.767899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b2bf028b-8b3e-44f2-b2da-31deeea913e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.228551 Jul 20 15:27:40.778773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1926/2853] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:40 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:40.876532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-e4f5eb1e-ea61-4d32-974e-f6a159f15f3f req-26b21f12-c972-4c9a-9028-50db1097b488 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:40.884823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-e4f5eb1e-ea61-4d32-974e-f6a159f15f3f req-26b21f12-c972-4c9a-9028-50db1097b488 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:40.885268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-e4f5eb1e-ea61-4d32-974e-f6a159f15f3f req-26b21f12-c972-4c9a-9028-50db1097b488 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:40.897515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-e4f5eb1e-ea61-4d32-974e-f6a159f15f3f req-26b21f12-c972-4c9a-9028-50db1097b488 service nova] Creating event network-changed:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:27:40.902990 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-e4f5eb1e-ea61-4d32-974e-f6a159f15f3f req-26b21f12-c972-4c9a-9028-50db1097b488 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029991 Jul 20 15:27:40.903950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1927/2854] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:41.787360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:41.788700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:41.796295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:41.797097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:41.797881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:41.798317 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.221085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:42.229377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:42.230845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:42.232538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-752d5099-e803-4123-85e5-044010f6fdd2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.449132 Jul 20 15:27:42.233191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1928/2855] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:41 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:27:42.237695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1127816965"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:42.238242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:42.243949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:42.244395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.245133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:42.245561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.380691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.network.security_group_api [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Adding security group 0245e78b-f532-40e7-a1a5-52b09cac1c15 to port bcb4295e-f687-475b-a18b-9de00faa1668 Jul 20 15:27:42.741967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2b416c65-4cb0-4eb3-8c3b-322588e35f8a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action" status: 202 len: 0 microversion: 2.1 time: 0.506962 Jul 20 15:27:42.742584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1929/2856] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:27:42 2022] POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action => generated 0 bytes in 509 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:27:42.757706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:42.759137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:42.776230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:42.776686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.778209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:42.778209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.948584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:42.956489 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d50468fd-2e0b-4083-b74c-9a6ce1111d28 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1757 microversion: 2.1 time: 0.209331 Jul 20 15:27:42.957048 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1930/2857] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:42 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1757 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:42.962445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-feadfc47-e27c-48ec-9a14-72560dafffd5 req-c1a35ec1-bb3a-409f-990c-d839c5e0e953 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:42.971474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-feadfc47-e27c-48ec-9a14-72560dafffd5 req-c1a35ec1-bb3a-409f-990c-d839c5e0e953 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:42.971925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-feadfc47-e27c-48ec-9a14-72560dafffd5 req-c1a35ec1-bb3a-409f-990c-d839c5e0e953 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:42.984120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-feadfc47-e27c-48ec-9a14-72560dafffd5 req-c1a35ec1-bb3a-409f-990c-d839c5e0e953 service nova] Creating event network-changed:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:27:42.989047 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-feadfc47-e27c-48ec-9a14-72560dafffd5 req-c1a35ec1-bb3a-409f-990c-d839c5e0e953 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030891 Jul 20 15:27:42.989934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1931/2858] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:43.251605 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:43.252296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:43.258686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:43.259161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:43.259942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:43.260372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:43.470933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:43.492363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:43.493899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:43.495080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9c8b769-d7e3-434e-8226-71c1faf0ca4c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.249021 Jul 20 15:27:43.495871 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1932/2859] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:43 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:44.516335 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:44.516895 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:44.524276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:44.524722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:44.525475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:44.525902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:44.660372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:44.668509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:44.669982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:44.671220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-58392dc1-2734-40db-83ac-16fef46f7dfe tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.160287 Jul 20 15:27:44.671776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1933/2860] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:44 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:45.691451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:45.696812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:45.708000 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:45.710911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:45.712768 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:45.713535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:45.873386 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:45.882099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:45.883634 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:45.884801 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-712d9de3-ca30-4a55-8201-b190214dc8c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.198902 Jul 20 15:27:45.885367 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1934/2861] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:45 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:45.890127 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "df4ce82f-a40d-44d4-b6af-47186a32f8f8"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:45.891263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:45.897724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:45.898169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:45.898951 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:45.899409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:45.916876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:45.994540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: POST /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments HTTP/1.1 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 74 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:27:45.996299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {"volumeAttachment": {"volumeId": "df4ce82f-a40d-44d4-b6af-47186a32f8f8"}} Jul 20 15:27:45.998031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': 'fbbcd35f-cd1f-4195-958f-1e9d1194aea0', 'body': {'volumeAttachment': {'volumeId': 'df4ce82f-a40d-44d4-b6af-47186a32f8f8'}}} Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:27:45.999012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:27:46.000677 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 400 to user: Invalid volume: volume df4ce82f-a40d-44d4-b6af-47186a32f8f8 is already attached to instances: fbbcd35f-cd1f-4195-958f-1e9d1194aea0 {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3477ce9e-cb0e-488d-9d9d-b411c8087e11 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.111356 Jul 20 15:27:46.003611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1935/2862] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:27:45 2022] POST /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 174 bytes in 112 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:27:46.060442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:46.060868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:46.067251 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.067699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:46.068449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.068882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:46.159588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid fbbcd35f-cd1f-4195-958f-1e9d1194aea0 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:27:46.234435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3a5ff82b-e4bb-4e6c-bb78-c76cd33cc418 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments/df4ce82f-a40d-44d4-b6af-47186a32f8f8" status: 202 len: 0 microversion: 2.1 time: 0.177806 Jul 20 15:27:46.235016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1936/2863] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:27:46 2022] DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments/df4ce82f-a40d-44d4-b6af-47186a32f8f8 => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:27:46.243595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:46.244609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:46.251855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.252607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:46.253479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.253991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:46.284810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3c494fae-f856-45c7-b2ba-6fd02822966c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045134 Jul 20 15:27:46.285534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1937/2864] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:46 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:46.907652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:46.908691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:46.919683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.920507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:46.921557 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:46.921974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:47.060620 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:47.069239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:47.070725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:47.071878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6810fb67-8246-47fc-b0a8-1029005040cb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.170972 Jul 20 15:27:47.072441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1938/2865] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:46 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:27:47.300690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:47.302223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:47.312026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:47.312475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:47.313224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:47.313653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:47.342961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f15548be-426c-4e4c-ab70-8f6816977196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048846 Jul 20 15:27:47.343535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1939/2866] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:47 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:48.089937 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:48.090556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:48.097945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:48.098390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:48.099205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:48.099640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:48.235991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:48.244301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:48.245772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:48.246931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb4e03aa-8db7-4769-9553-92ec0ae1f7bd tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.160952 Jul 20 15:27:48.247494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1940/2867] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:48 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:48.358694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:48.359648 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:48.366013 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:48.366445 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:48.367259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:48.367696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:48.398369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d2e2935c-f174-48d8-8c66-821f4e8f760a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046263 Jul 20 15:27:48.398973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1941/2868] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:48 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:49.269519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:49.270610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:49.281031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:49.281461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:49.282249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:49.282699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:49.416144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:49.424199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:49.425683 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:49.426831 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-14d372b8-334a-44f4-b4f1-7538325401a2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.163900 Jul 20 15:27:49.427414 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1942/2869] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:49 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:49.431963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:49.432869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:49.438615 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:49.439100 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:49.439832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:49.440259 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:49.478376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9bd0186f-35cc-4c3c-a112-aad4f82af7a4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049893 Jul 20 15:27:49.479082 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1943/2870] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:49 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:50.448527 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:50.449672 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:50.452857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=92 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:50.458400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:50.458848 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:50.459662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:50.460092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:50.602498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:50.612449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:50.613965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:50.615150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-036dfa71-74a3-4988-a0a1-b1770c0fcc55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.173086 Jul 20 15:27:50.615713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1944/2871] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:50 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:50.620186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:50.621094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:50.626950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:50.627426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:50.628162 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:50.628593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:50.657528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-019a20ec-34bb-46af-9d92-10023a5dcf6d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040829 Jul 20 15:27:50.658070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1945/2872] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:50 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:51.637083 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:51.638058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:51.649461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:51.650236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:51.651585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:51.652274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:51.789843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:51.798609 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:51.800178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:51.801350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d2f450d-4610-4263-88c8-bb6af9cfb6cf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.171027 Jul 20 15:27:51.801953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1946/2873] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:51 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:51.806454 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:51.807392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:51.813846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:51.814286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:51.815071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:51.815529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:51.846694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-45d91a5e-19fc-4021-b742-7b7a2bceb0a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043723 Jul 20 15:27:51.847294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1947/2874] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:51 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:51.911849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-0649bb14-2c2d-41c2-a990-c5b879464453 req-c8311918-db4d-4e93-bd2b-3fdd61c294e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "name": "network-vif-plugged", "status": "completed", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:51.920600 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-0649bb14-2c2d-41c2-a990-c5b879464453 req-c8311918-db4d-4e93-bd2b-3fdd61c294e7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:51.921071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-0649bb14-2c2d-41c2-a990-c5b879464453 req-c8311918-db4d-4e93-bd2b-3fdd61c294e7 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:51.932740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-0649bb14-2c2d-41c2-a990-c5b879464453 req-c8311918-db4d-4e93-bd2b-3fdd61c294e7 service nova] Creating event network-vif-plugged:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:27:51.937510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-0649bb14-2c2d-41c2-a990-c5b879464453 req-c8311918-db4d-4e93-bd2b-3fdd61c294e7 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029905 Jul 20 15:27:51.938099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1948/2875] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:52.825629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:52.826725 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:52.833377 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:52.833816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:52.834572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:52.834997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:52.972839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:52.984874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:52.987286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:52.989114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7ef60d73-0c86-4e05-a549-3f8b4e974942 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.171098 Jul 20 15:27:52.989882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1949/2876] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:52 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:52.994392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:52.995372 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:53.001556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.001997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.002757 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.003229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.033043 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e2e4569d-009f-415d-8f60-e29f5805605d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042072 Jul 20 15:27:53.033709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1950/2877] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:52 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:53.454966 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:53.455926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:53.462240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.462691 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.463479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.463917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.493922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a4c52ff1-e021-406d-b48e-978e5655480e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.042929 Jul 20 15:27:53.494481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1951/2878] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:53 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 371 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:27:53.643266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:53.643676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:53.650655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.651229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.652403 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.652555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.661480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:53.692501 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c8713030-20ee-481e-82b4-de67eb14d8c3 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb" status: 200 len: 194 microversion: 2.1 time: 0.054225 Jul 20 15:27:53.693096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1952/2879] 10.222.0.15 () {58 vars in 1412 bytes} [Wed Jul 20 15:27:53 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb => generated 194 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:53.702695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:53.703270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:53.710005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.710455 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.711243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:53.711674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:53.794003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'services' on Instance uuid 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:27:53.872601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e14cb2a-563e-4cff-90b6-9422ace6715d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb" status: 202 len: 0 microversion: 2.1 time: 0.173790 Jul 20 15:27:53.873418 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1953/2880] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:27:53 2022] DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/0ace56e1-fae7-49c2-82a7-0eaa38b02afb => generated 0 bytes in 175 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:27:54.008348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:54.009039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:54.016062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:54.016513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:54.017277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:54.017743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:54.155257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:54.163112 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:54.164599 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:54.165755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c35eec11-5caa-4e6f-89e2-101a0968191e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.161658 Jul 20 15:27:54.166349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1954/2881] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:54 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:54.170863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:54.171861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:54.177945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:54.178390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:54.179172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:54.179598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:54.207955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4ee34e9c-b1a8-46ad-adb2-e654448b4c79 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040605 Jul 20 15:27:54.208473 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1955/2882] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:54 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:55.187760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:55.188417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:55.195201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.195651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.196405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.196841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.340577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:55.348488 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:55.349981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:55.351171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b9166349-8e60-46c9-81af-b474cc2c5a6a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1746 microversion: 2.20 time: 0.169311 Jul 20 15:27:55.351733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1956/2883] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:55 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1746 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:55.356689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:55.358016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:55.364054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.364506 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.365254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.365702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.394588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-159e7daa-c0f8-4f1e-96b4-b03051fbb9db tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041826 Jul 20 15:27:55.395155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1957/2884] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:55 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 15:27:55.452009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:55.452425 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:55.458537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.458979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.459753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.460178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.487849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:27:55.573554 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea98c328-45a6-4a0e-a450-03807c3124d8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 204 len: 0 microversion: 2.1 time: 0.125108 Jul 20 15:27:55.574312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1958/2885] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:27:55 2022] DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:27:55.585229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:55.586106 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:55.593485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.593934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.594789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:55.595254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:55.748128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:55.762199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6c1187b7-fc23-450f-962d-56190d060fe7 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1678 microversion: 2.1 time: 0.181608 Jul 20 15:27:55.775538 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1959/2886] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:55 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1678 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:27:56.068808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:56.069297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:56.076007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.076482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.077235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.077851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.113155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-26765c3a-2f8b-4352-9a2a-307188290ea7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/cb10eea1-ff88-47da-af68-7a13027e52a5" status: 200 len: 194 microversion: 2.1 time: 0.050382 Jul 20 15:27:56.113777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1960/2887] 10.222.0.15 () {58 vars in 1412 bytes} [Wed Jul 20 15:27:56 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/cb10eea1-ff88-47da-af68-7a13027e52a5 => generated 194 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:56.126191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:56.126680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:56.132915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.133326 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.134077 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.134530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.267690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'services' on Instance uuid 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:27:56.354462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aed8357e-fcf0-4455-9805-55b4b8ce142e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/cb10eea1-ff88-47da-af68-7a13027e52a5" status: 202 len: 0 microversion: 2.1 time: 0.232728 Jul 20 15:27:56.355256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1961/2888] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:27:56 2022] DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments/cb10eea1-ff88-47da-af68-7a13027e52a5 => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:27:56.372128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:56.372968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:56.380556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.381140 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.382004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.382535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.528287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:56.537276 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:56.538812 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:56.540044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-39bf7e25-1c32-47aa-abdc-19890c619159 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.172937 Jul 20 15:27:56.548752 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1962/2889] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:56 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:27:56.783180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:56.783733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:56.791316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.792020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.793080 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:56.793669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:56.954799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:56.963494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0ae49a69-f412-4057-a7f5-ce05eb5d2ee6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1678 microversion: 2.1 time: 0.185254 Jul 20 15:27:56.972564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1963/2890] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:56 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1678 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:57.560345 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:57.561388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:57.568314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:57.568700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:57.569486 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:57.569934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:57.719165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:57.729578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:57.731225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:57.733448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2d44f562-1b45-47fe-b359-f3ee6f2e6144 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.177285 Jul 20 15:27:57.733448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1964/2891] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:57 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:27:57.982622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:57.983410 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:57.991549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:57.991994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:57.993012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:57.993535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:58.155746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:58.159888 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:27:58.171189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bb7a634d-37ab-4756-8dda-9460eb82ab46 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1678 microversion: 2.1 time: 0.189544 Jul 20 15:27:58.178435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1965/2892] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:57 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1678 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:58.752742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:58.753443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:58.769375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:58.770008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:58.770880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:58.771662 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.016495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:59.027837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:59.043036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:59.044442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f63c9a0c-4018-4184-80ba-ef5e916160e0 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.295968 Jul 20 15:27:59.058263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1966/2893] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:27:58 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:27:59.185670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:59.186366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:59.194511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.195321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.196970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.197525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.305185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:27:59.314207 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9c9c036b-da1a-489d-9d8d-14de1e9a0753 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 200 len: 1610 microversion: 2.1 time: 0.132481 Jul 20 15:27:59.323793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1967/2894] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:27:59 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 1610 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:27:59.347802 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-545bb500-fa96-47dc-aa61-d8b9b8895095 req-740ec559-3462-4a78-954f-79c33f3ed1ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbbcd35f-cd1f-4195-958f-1e9d1194aea0", "name": "network-vif-deleted", "tag": "c05d9926-8cc3-4a5e-8246-68dd7a0f2770"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:27:59.356495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-545bb500-fa96-47dc-aa61-d8b9b8895095 req-740ec559-3462-4a78-954f-79c33f3ed1ab service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.357442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-545bb500-fa96-47dc-aa61-d8b9b8895095 req-740ec559-3462-4a78-954f-79c33f3ed1ab service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.375635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-545bb500-fa96-47dc-aa61-d8b9b8895095 req-740ec559-3462-4a78-954f-79c33f3ed1ab service nova] Creating event network-vif-deleted:c05d9926-8cc3-4a5e-8246-68dd7a0f2770 for instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 on hv-cinder-84902 Jul 20 15:27:59.381826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-545bb500-fa96-47dc-aa61-d8b9b8895095 req-740ec559-3462-4a78-954f-79c33f3ed1ab service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039692 Jul 20 15:27:59.382938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1968/2895] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:27:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:27:59.682820 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:59.683904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:59.690551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.691034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.691767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.692199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.724806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-59902a66-53d7-4715-a775-9df24d04a3a1 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046166 Jul 20 15:27:59.730058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1969/2896] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:59 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 15:27:59.859840 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:27:59.860762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:27:59.875584 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.876029 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.876794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:27:59.877224 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:27:59.932837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2edbec9b-7103-4778-9145-825cacff982d tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.077089 Jul 20 15:27:59.933822 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1970/2897] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:27:59 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae/os-volume_attachments => generated 25 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 15:28:00.009384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:00.010216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:00.031928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.033484 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.034724 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.035371 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.074300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:28:00.184465 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4905a8d6-6407-4713-ac76-49c57be6dbb8 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 204 len: 0 microversion: 2.1 time: 0.181833 Jul 20 15:28:00.190298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1971/2898] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:28:00 2022] DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 0 bytes in 185 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 20 15:28:00.198022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:00.198907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:00.206658 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.207411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.208338 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.211717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.394146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:00.406305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:00.407701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:00.408949 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-42f9ec92-b666-4245-bff3-a2c330f8b2f9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.216507 Jul 20 15:28:00.423466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1972/2899] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:00 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:00.424794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:00.425461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:00.434833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.435295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.436136 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.436572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.604719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:00.613328 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fd18d4ff-6fa0-46f4-ab47-2cb18b90108a tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1673 microversion: 2.1 time: 0.192243 Jul 20 15:28:00.624787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1973/2900] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:00 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1673 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:00.624787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:00.624787 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:00.625238 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.626775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.626775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:00.626989 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:00.642692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 HTTP/1.1 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:00.644723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:28:00.646915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'fbbcd35f-cd1f-4195-958f-1e9d1194aea0'} Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:00.648032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:28:00.649995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:00.651610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:00.651610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0e8310cf-e5e1-4595-8dde-2c520e2d08a2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 404 len: 111 microversion: 2.1 time: 0.031279 Jul 20 15:28:00.651610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1974/2901] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:00 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:28:00.656393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b05d3f8a-c769-417d-a121-8957849ddfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '.version_select at 0x7f7b15cc73a0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:00.680150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b05d3f8a-c769-417d-a121-8957849ddfde tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-341583278" status: 202 len: 0 microversion: 2.1 time: 0.027478 Jul 20 15:28:00.681283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1975/2902] 10.222.0.15 () {58 vars in 1217 bytes} [Wed Jul 20 15:28:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-341583278 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:28:01.433184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:01.434586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:01.442235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:01.442908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:01.443930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:01.444528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:01.622645 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:01.637520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:01.640664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:01.643134 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1d55a94e-1039-4891-9c2e-b84050de2f89 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.215022 Jul 20 15:28:01.654441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1976/2903] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:01 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:01.654441 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-524e3c41-cde6-417b-9cbd-d8cebdb1a84c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: .version_select at 0x7f7b160023a0>, body: {"keypair": {"name": "tempest-keypair-26907741", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFrFbuZD9RVNZ1Md53oCUKg2+yIyoJsQAAWZ0Zc1NBN34DLWGPiN8HjgSgeg7TJlyDAnCHu8zjb2+4MGoJx7mNJSCKfCYpsSB7TvfYDWqkKQzIrCKl2qbgKy+W0CmGbCjQ=="}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:01.672304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-524e3c41-cde6-417b-9cbd-d8cebdb1a84c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:28:01.678655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-524e3c41-cde6-417b-9cbd-d8cebdb1a84c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'key_pairs'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:28:01.707954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-524e3c41-cde6-417b-9cbd-d8cebdb1a84c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.062114 Jul 20 15:28:01.708515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1977/2904] 10.222.0.15 () {60 vars in 1158 bytes} [Wed Jul 20 15:28:01 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:01.713890 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:01.714800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:01.722914 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:01.723392 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:01.724163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:01.724593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:01.933021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:01.942879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3f78d4c3-4f0f-447b-aa96-892ff9aace7e tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1673 microversion: 2.1 time: 0.233267 Jul 20 15:28:01.944202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1978/2905] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:01 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1673 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:02.662707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:02.664796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:02.672277 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:02.673074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:02.674154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:02.674968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:02.983946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:02.994199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:02.996035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:02.997518 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebe92c39-4719-47bf-b091-7ecba82359c9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.339367 Jul 20 15:28:02.998152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1979/2906] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:02 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:03.014723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:03.015234 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:03.021814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:03.022263 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:03.023030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:03.023491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:03.314836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:03.325494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1a9f8464-1e10-4a9e-83ee-5eb5a955771c tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1673 microversion: 2.1 time: 0.315319 Jul 20 15:28:03.335696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1980/2907] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:02 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1673 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:04.016669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:04.017498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:04.024466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:04.024915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:04.025694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:04.026168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:04.310181 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:04.322405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:04.324334 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:04.325510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dd0e154d-a859-494e-894b-558dc8c689f3 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.313208 Jul 20 15:28:04.326101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1981/2908] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:04 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:04.336449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1779267889", "imageRef": "6af58e02-ad76-42f6-8f03-0d36ffc885a4", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "bb36c589-f60d-4c91-8345-5777287e4f0d"}], "security_groups": [{"name": "tempest-securitygroup--1504336125"}], "key_name": "tempest-keypair-26907741", "user_data": "}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:04.344146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:04.502664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.network.neutron [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] validate_networks() for [('bb36c589-f60d-4c91-8345-5777287e4f0d', None, None, None, None, None)] {{(pid=124396) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 20 15:28:05.118482 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 20 15:28:05.119178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image limits 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 20 15:28:05.119437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Flavor pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 20 15:28:05.119847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Image pref 0:0:0 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 20 15:28:05.120283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.virt.hardware [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=124396) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 20 15:28:05.130169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.130881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.131921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.132690 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.156630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 20 15:28:05.164679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.quota [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Getting quotas for user e3574b5b8f3241b192b0b810122d4769 and project 5d984345e0d340818c9030f70bace6d3. Resources: {'cores', 'ram', 'instances'} {{(pid=124396) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 20 15:28:05.180689 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Going to run 1 instances... {{(pid=124396) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 20 15:28:05.182980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='6af58e02-ad76-42f6-8f03-0d36ffc885a4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=124396) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 20 15:28:05.252155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-df7600a9-8d71-483f-a5d2-9d2a1693549c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.921354 Jul 20 15:28:05.252792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1982/2909] 10.222.0.15 () {60 vars in 1146 bytes} [Wed Jul 20 15:28:04 2022] POST /compute/v2.1/servers => generated 402 bytes in 923 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 20 15:28:05.257466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:05.258244 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:05.264904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.265348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.266298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.266907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.299536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.299986 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.300461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lazy-loading 'fault' on Instance uuid 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:28:05.346350 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5f51124af1e846fb89552078c81afdb0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b966dbac18614c979f674666ba817def', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:05.358618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-aab28465-6df6-4b70-b6e8-260743394743 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 200 len: 1327 microversion: 2.1 time: 0.104348 Jul 20 15:28:05.359955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1983/2910] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:05 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 1327 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:05.368565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-f998befd-27dc-42e7-beea-f4fc51592737 req-5f3b272e-ea94-4f7e-8eab-f7bfb595d335 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae", "name": "network-vif-deleted", "tag": "b9ddd43f-d051-4dea-8402-f48e7e16421e"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:05.377832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f998befd-27dc-42e7-beea-f4fc51592737 req-5f3b272e-ea94-4f7e-8eab-f7bfb595d335 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.378555 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-f998befd-27dc-42e7-beea-f4fc51592737 req-5f3b272e-ea94-4f7e-8eab-f7bfb595d335 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.404809 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-f998befd-27dc-42e7-beea-f4fc51592737 req-5f3b272e-ea94-4f7e-8eab-f7bfb595d335 service nova] Creating event network-vif-deleted:b9ddd43f-d051-4dea-8402-f48e7e16421e for instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae on hv-cinder-84902 Jul 20 15:28:05.410321 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-f998befd-27dc-42e7-beea-f4fc51592737 req-5f3b272e-ea94-4f7e-8eab-f7bfb595d335 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049105 Jul 20 15:28:05.410922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1984/2911] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:05.416253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-0377f279-f7df-4727-ae9e-7f9ad865cf9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:05.416816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-0377f279-f7df-4727-ae9e-7f9ad865cf9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:05.477423 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-0377f279-f7df-4727-ae9e-7f9ad865cf9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:05.486412 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-0377f279-f7df-4727-ae9e-7f9ad865cf9e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1240 microversion: 2.1 time: 0.074206 Jul 20 15:28:05.487383 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1985/2912] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:05 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1240 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:05.500072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:05.500693 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:05.507167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.507619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.508395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:05.508825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:05.513828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:05.650950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:05.659638 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:05.661116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:05.662300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-880d10dd-6dc9-4e98-b8fb-06eaad966726 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1741 microversion: 2.20 time: 0.166197 Jul 20 15:28:05.662866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1986/2913] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:05 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1741 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:06.376823 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:06.377474 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:06.384438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:06.384889 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:06.385652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:06.386120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:06.400130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae HTTP/1.1 Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:06.401789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:06.403553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae'} Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:06.404570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:06.406088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:06.407636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:06.407636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6ab6a03a-2718-4347-bb42-cde52dde069b tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 404 len: 111 microversion: 2.1 time: 0.030243 Jul 20 15:28:06.407636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1987/2914] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:06 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:28:06.412012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f9aa5a34-350b-46eb-9768-fff013cfe9dd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '.version_select at 0x7f7b15cfb8b0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:06.430967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f9aa5a34-350b-46eb-9768-fff013cfe9dd tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-179996137" status: 202 len: 0 microversion: 2.1 time: 0.022468 Jul 20 15:28:06.431573 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1988/2915] 10.222.0.15 () {58 vars in 1217 bytes} [Wed Jul 20 15:28:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-179996137 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:28:06.503661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dabc4978-fc8f-4576-aeeb-ec0851bd3420 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:06.504581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dabc4978-fc8f-4576-aeeb-ec0851bd3420 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:06.565110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dabc4978-fc8f-4576-aeeb-ec0851bd3420 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:06.590704 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dabc4978-fc8f-4576-aeeb-ec0851bd3420 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1240 microversion: 2.1 time: 0.090914 Jul 20 15:28:06.591759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1989/2916] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:06 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1240 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:06.680984 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:06.681587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:06.689344 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:06.690138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:06.691107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:06.691534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:06.961487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:06.969814 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:06.971463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:06.972594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5e1f6b9a-6b50-4dbd-a7af-1ec97c9a7910 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1723 microversion: 2.20 time: 0.296075 Jul 20 15:28:06.973168 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1990/2917] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:06 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1723 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:07.608247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:07.608892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:07.615920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:07.616364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:07.617123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:07.617553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:07.684804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:07.692832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ba90ff41-0662-42da-a43c-62de6dd5fec2 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1244 microversion: 2.1 time: 0.088789 Jul 20 15:28:07.693443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1991/2918] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:07 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1244 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:07.992302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:07.992916 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:07.999529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:07.999891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:08.000640 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:08.001062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:08.138967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:08.147449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:08.148941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:08.150097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-689465cd-bb2e-4922-b246-1ac09c342af9 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1723 microversion: 2.20 time: 0.162409 Jul 20 15:28:08.150653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1992/2919] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:07 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:08.155252 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-87bc10f2-7119-4c22-b7ba-3a8e7d6859e9 req-f18b87cf-7060-4272-89fd-28ea5bdf3255 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "name": "network-vif-unplugged", "status": "completed", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:08.163623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-87bc10f2-7119-4c22-b7ba-3a8e7d6859e9 req-f18b87cf-7060-4272-89fd-28ea5bdf3255 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:08.164059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-87bc10f2-7119-4c22-b7ba-3a8e7d6859e9 req-f18b87cf-7060-4272-89fd-28ea5bdf3255 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:08.174759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-87bc10f2-7119-4c22-b7ba-3a8e7d6859e9 req-f18b87cf-7060-4272-89fd-28ea5bdf3255 service nova] Creating event network-vif-unplugged:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:28:08.179719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-87bc10f2-7119-4c22-b7ba-3a8e7d6859e9 req-f18b87cf-7060-4272-89fd-28ea5bdf3255 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.027671 Jul 20 15:28:08.180298 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1993/2920] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:08 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:08.715200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:08.716274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:08.723706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:08.724146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:08.724902 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:08.725327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:08.789214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:08.800509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-94bf3ca8-0b02-461e-add4-650e14adf0e1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1244 microversion: 2.1 time: 0.092316 Jul 20 15:28:08.801193 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1994/2921] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:08 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1244 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:09.169806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:09.170442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:09.177272 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.178308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.179417 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.180282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.320092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:09.328243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:09.329723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:09.330879 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-54e9d904-a061-4056-bc11-349fd9386260 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1726 microversion: 2.20 time: 0.166438 Jul 20 15:28:09.331511 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1995/2922] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:09 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1726 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:09.348621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "01d86931-d062-49a1-9bf3-9e2b757f477c"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:09.349653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:09.356265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.357079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.357830 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.358264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.646852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-588926ef-efab-4554-9dae-ee733b9d9127 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.302254 Jul 20 15:28:09.652135 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1996/2923] 10.222.0.15 () {62 vars in 1363 bytes} [Wed Jul 20 15:28:09 2022] POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments => generated 188 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 20 15:28:09.652663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:09.653155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:09.659851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.660295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.661055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.661781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.675722 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae HTTP/1.1 Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:09.677540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.679646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae'} Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:09.680669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.682120 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.683711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:09.683711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4e5d4291-dea2-4fe9-acbd-4207a063eac2 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 404 len: 111 microversion: 2.1 time: 0.030418 Jul 20 15:28:09.683711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1997/2924] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:28:09 2022] DELETE /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:28:09.687329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:09.687935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:09.694659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.695173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.696246 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.696681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.712065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae HTTP/1.1 Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:09.713580 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.715855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae'} Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:09.717205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. Jul 20 15:28:09.719229 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:09.720734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:09.720734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ab979523-b64a-4fb2-8cd9-6ddf06fa16c7 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae" status: 404 len: 111 microversion: 2.1 time: 0.031284 Jul 20 15:28:09.720734 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1998/2925] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:09 2022] GET /compute/v2.1/servers/4d3ebdf4-d817-4409-b4ea-e5f0b55f1eae => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:28:09.772184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:09.772612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:09.780792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.781242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:09.782067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:09.782490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.092089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9dd473b1-d469-49f8-acf4-17e8546f8c4d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments/01d86931-d062-49a1-9bf3-9e2b757f477c" status: 202 len: 0 microversion: 2.20 time: 0.324492 Jul 20 15:28:10.092766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 1999/2926] 10.222.0.15 () {60 vars in 1456 bytes} [Wed Jul 20 15:28:09 2022] DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments/01d86931-d062-49a1-9bf3-9e2b757f477c => generated 0 bytes in 326 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 20 15:28:10.110324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:10.110939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:10.129934 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.130389 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.131184 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.133788 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.218467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:10.226535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-af37653b-cf52-4572-800f-36d44f8f3682 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1310 microversion: 2.1 time: 0.120369 Jul 20 15:28:10.236709 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2000/2927] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:10 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1310 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:10.241665 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:10.242264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:10.248782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.249494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.250257 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.250785 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.332481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=86,nova_cell1:INSERT=5,nova_cell1:UPDATE=9 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:10.360008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9968c8ba-9615-4020-8b82-54f9310f5f6f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/action" status: 202 len: 0 microversion: 2.20 time: 0.122021 Jul 20 15:28:10.360687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2001/2928] 10.222.0.15 () {62 vars in 1318 bytes} [Wed Jul 20 15:28:10 2022] POST /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/action => generated 0 bytes in 123 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:28:10.368936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:10.369547 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:10.375811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.376264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.377026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.377462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.587786 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:10.598561 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:10.600159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:10.601358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-35569716-528d-4025-be21-4b8e407f9832 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1674 microversion: 2.20 time: 0.236102 Jul 20 15:28:10.602089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2002/2929] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:10 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1674 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:10.858589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:10.859211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:10.865206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.865717 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.866424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.866859 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.881928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 HTTP/1.1 Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:10.883539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.885188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '52220445-7cc1-4635-9566-c6fd58b1fc96'} Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:10.886060 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.887694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.889253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:10.889253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c528a34a-a040-4e5d-a791-89b6cba2ba36 tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 404 len: 111 microversion: 2.1 time: 0.031120 Jul 20 15:28:10.889253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2003/2930] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:28:10 2022] DELETE /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:28:10.894692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:10.895333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] [instance: 52220445-7cc1-4635-9566-c6fd58b1fc96] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:10.902490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.903171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.904529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:10.905227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:10.922708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] HTTP exception thrown: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Request method failure captured: Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 HTTP/1.1 Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWni91cwSehbg3dv3WVIv9txuMKq4S01ILJNI7rH9PLeK9tyZdcqAGnP2CZ4lhn2RTTh9QB-YadKhl_d8HbEx00hACaSnjw815QBKqUlGKVCdnNJpedO1aZKTTLrRPghQUCseYINXgFviOta78bsT_1MLy5SvuuUeJxnv0KLccptXg Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:28:10.924595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_b966dbac18614c979f674666ba817def"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/b966dbac18614c979f674666ba817def"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/b966dbac18614c979f674666ba817def"}]}] Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: b966dbac18614c979f674666ba817def Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1439415870 Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 5f51124af1e846fb89552078c81afdb0 Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeTestJSON-1439415870-project Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.926466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '52220445-7cc1-4635-9566-c6fd58b1fc96'} Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:28:10.927543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. Jul 20 15:28:10.929240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:28:10.931385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] Returning 404 to user: Instance 52220445-7cc1-4635-9566-c6fd58b1fc96 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:28:10.932404 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-256b72dc-62c5-4960-91e5-cc982868b8eb tempest-AttachVolumeTestJSON-1439415870 tempest-AttachVolumeTestJSON-1439415870-project] 10.222.0.15 "GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96" status: 404 len: 111 microversion: 2.1 time: 0.042752 Jul 20 15:28:10.936630 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2004/2931] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:10 2022] GET /compute/v2.1/servers/52220445-7cc1-4635-9566-c6fd58b1fc96 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:28:11.248513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:11.253384 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:11.265715 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:11.266188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:11.267881 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:11.268312 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:11.501696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:11.511078 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e2e5f8bb-8c44-4896-979d-8258679d371a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.270559 Jul 20 15:28:11.512086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2005/2932] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:11 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:11.621287 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:11.622218 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:11.629163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:11.629993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:11.630962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:11.631590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:11.876322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:11.887875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:11.889416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:11.890781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-578d2fcc-ff8e-4f32-8be6-15dcd1bd4c88 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1676 microversion: 2.20 time: 0.273938 Jul 20 15:28:11.891585 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2006/2933] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:11 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1676 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:11.994068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-d720b9f3-a626-48e0-806b-803af8175eb5 req-add0025b-1224-4299-bdd8-c56d1f732a8b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4464f0b3-f153-4575-9588-8bc7a46dadd5", "tag": "47f645c2-c472-46a5-ad91-9d245af7f214"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:12.008302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d720b9f3-a626-48e0-806b-803af8175eb5 req-add0025b-1224-4299-bdd8-c56d1f732a8b service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:12.008783 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-d720b9f3-a626-48e0-806b-803af8175eb5 req-add0025b-1224-4299-bdd8-c56d1f732a8b service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:12.020832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-d720b9f3-a626-48e0-806b-803af8175eb5 req-add0025b-1224-4299-bdd8-c56d1f732a8b service nova] Creating event network-changed:47f645c2-c472-46a5-ad91-9d245af7f214 for instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 on hv-cinder-84902 Jul 20 15:28:12.028095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-d720b9f3-a626-48e0-806b-803af8175eb5 req-add0025b-1224-4299-bdd8-c56d1f732a8b service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038500 Jul 20 15:28:12.029470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2007/2934] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:12.529589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:12.530285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:12.536845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:12.537283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:12.538236 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:12.538837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:13.007079 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:13.017026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c7af4a7c-c1c8-472f-bb6e-9fdc44eab14f tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.491610 Jul 20 15:28:13.018202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2008/2935] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:12 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:13.023875 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:13.024669 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:13.031460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:13.032150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:13.033478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:13.034125 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:13.378760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:13.404512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:13.406187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:13.412943 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72b56140-4204-4405-a6b5-ca7c6091aea8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.393409 Jul 20 15:28:13.413987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2009/2936] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:13 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:14.034419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:14.035806 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:14.043450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:14.043899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:14.044664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:14.045094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:14.272363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:14.281731 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-83520744-0c72-4bf5-ac46-a8a522550a7b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.251679 Jul 20 15:28:14.284963 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2010/2937] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:14 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:14.431535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:14.432150 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:14.439288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:14.440008 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:14.441288 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:14.441909 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:14.703200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:14.716946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:14.720438 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:14.722437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7c833597-d59b-468a-8251-720c4b4ab36e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.294686 Jul 20 15:28:14.735825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2011/2938] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:14 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:15.299115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:15.299766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:15.302095 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=2 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:15.307939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:15.308393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:15.309149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:15.309577 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:15.447976 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:15.456419 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-312720f9-1d86-4064-94f1-9151e7a3f2d6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.161069 Jul 20 15:28:15.456991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2012/2939] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:15 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:15.743138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:15.743767 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:15.749921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:15.750359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:15.751142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:15.751639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:15.882563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:15.891462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:15.893035 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:15.894275 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2048eb95-4ba5-4c3f-9bf8-6628bc1259bb tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.155083 Jul 20 15:28:15.894919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2013/2940] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:15 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:16.473031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:16.473655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:16.484878 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:16.490071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:16.490911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:16.491380 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:16.590304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell0:SELECT=14 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:16.782225 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:16.793375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b975ead0-e598-4af4-987b-daa8a44e33f6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.324179 Jul 20 15:28:16.803126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2014/2941] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:16 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:16.913863 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:16.914463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:16.920839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:16.921297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:16.922062 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:16.922497 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:17.124216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:17.132933 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:17.134743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:17.136147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8e2261ed-5bc3-4195-859b-abcf069a7e55 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.226607 Jul 20 15:28:17.137199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2015/2942] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:16 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:17.814635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:17.815319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:17.823432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:17.823858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:17.824625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:17.825053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:18.014825 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:18.023348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ffe471be-15d4-4f9e-aa41-4838051215f3 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.214267 Jul 20 15:28:18.024192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2016/2943] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:17 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:18.161296 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:18.161904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:18.168882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:18.170548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:18.172594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:18.173239 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:18.305647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:18.314732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:18.316572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:18.318101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4c9b41e8-4e7c-43df-9cba-f9b9d3abffdf tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.160505 Jul 20 15:28:18.318647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2017/2944] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:18 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:19.041967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:19.043446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:19.050507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:19.050954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:19.051745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:19.052209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:19.224151 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:19.233970 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5dfbd516-fa8d-440f-ab43-65e4dd49f19d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.195921 Jul 20 15:28:19.234663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2018/2945] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:19 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:19.336588 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:19.337196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:19.339069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:19.344635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:19.345159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:19.346015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:19.346702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:19.588702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:19.597157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:19.599492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:19.601009 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-294ae41a-93ae-4b1d-b033-8324047664af tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.268209 Jul 20 15:28:19.602156 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2019/2946] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:19 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:20.251666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:20.252314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:20.260046 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:20.260924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:20.262142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:20.262837 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:20.268178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:20.432101 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:20.456505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-099d20ca-2d96-42b6-95b8-44c24d330d5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.208946 Jul 20 15:28:20.465857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2020/2947] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:20 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:20.472521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-18689169-59ea-49fa-baa6-f5e2b805d414 req-50b30926-9c63-4e88-a00c-2d976e90c9fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4464f0b3-f153-4575-9588-8bc7a46dadd5", "name": "network-vif-plugged", "status": "completed", "tag": "47f645c2-c472-46a5-ad91-9d245af7f214"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:20.500182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-18689169-59ea-49fa-baa6-f5e2b805d414 req-50b30926-9c63-4e88-a00c-2d976e90c9fb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:20.500702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-18689169-59ea-49fa-baa6-f5e2b805d414 req-50b30926-9c63-4e88-a00c-2d976e90c9fb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:20.524076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-18689169-59ea-49fa-baa6-f5e2b805d414 req-50b30926-9c63-4e88-a00c-2d976e90c9fb service nova] Creating event network-vif-plugged:47f645c2-c472-46a5-ad91-9d245af7f214 for instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 on hv-cinder-84902 Jul 20 15:28:20.530088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-18689169-59ea-49fa-baa6-f5e2b805d414 req-50b30926-9c63-4e88-a00c-2d976e90c9fb service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062652 Jul 20 15:28:20.530699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2021/2948] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:20.620595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:20.621302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:20.627981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:20.628642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:20.629746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:20.635492 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:20.789699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:20.803249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:20.803249 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:20.804623 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a8bd46f1-aeef-4581-a5fb-0affe3f008db tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.188427 Jul 20 15:28:20.817320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2022/2949] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:20 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:21.474470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:21.475885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:21.488639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:21.489086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:21.500054 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:21.500519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:21.647217 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:21.658962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-70ce7de8-6b31-434f-8a81-08f73daa7ad1 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.189339 Jul 20 15:28:21.660010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2023/2950] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:21 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:21.827953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:21.828517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:21.835103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:21.835556 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:21.836540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:21.837152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:21.983892 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:22.004322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:22.008590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:22.010869 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-73656315-a78b-48f5-8e47-cf05e823ba83 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.187409 Jul 20 15:28:22.011652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2024/2951] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:21 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:22.676483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:22.677270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:22.695843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:22.696299 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:22.697297 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:22.697730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:22.842365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:22.850270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dcbf3b61-2d28-476c-8947-7931720819a0 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.177908 Jul 20 15:28:22.850850 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2025/2952] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:22 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:23.034930 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:23.035747 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:23.042838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:23.043862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:23.044905 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:23.045563 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:23.234443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:23.252260 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:23.253746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:23.254907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b0d6eee6-9d4c-40c8-94fa-6c638ae1f2dc tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.224663 Jul 20 15:28:23.266070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2026/2953] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:23 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:23.868056 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:23.868853 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:23.875701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:23.876142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:23.876954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:23.877364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:24.072114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:24.084678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9092fdde-2fcc-4a92-ae55-6b0c9262290c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.220399 Jul 20 15:28:24.085664 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2027/2954] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:23 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:24.274463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:24.275625 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:24.283322 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:24.283946 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:24.284642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:24.285058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:24.438921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:24.446876 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:24.448391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:24.449700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-288839d9-ce23-4d5d-88e7-2e2d807a6f86 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.180575 Jul 20 15:28:24.450499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2028/2955] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:24 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:25.102015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:25.103657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:25.111256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:25.111637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:25.112406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:25.112868 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:25.271843 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:25.287695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d0b63f8b-a200-45af-8917-bda9692445ca tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.187743 Jul 20 15:28:25.287695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2029/2956] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:25 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:25.469359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:25.469929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:25.477163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:25.477849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:25.478861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:25.479495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:25.628055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:25.636833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:25.638385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:25.639590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-62bdd1e6-347b-4c93-aa71-1538a2182276 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.175241 Jul 20 15:28:25.640154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2030/2957] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:25 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:26.305382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:26.306137 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:26.313406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:26.313841 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:26.314603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:26.315026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:26.455529 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:26.466452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3a26d06-1dc7-406c-afe6-db12c01ea8fb tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.164390 Jul 20 15:28:26.467286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2031/2958] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:26 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:26.657824 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:26.658431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:26.666360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:26.667681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:26.668931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:26.669546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:26.899745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:26.909800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:26.911782 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:26.923833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6f116d91-0670-4cc9-95bf-dca518fb442d tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.269804 Jul 20 15:28:26.926220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2032/2959] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:26 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:27.486364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:27.488985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:27.495954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:27.496400 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:27.497159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:27.497635 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:27.634070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:27.645378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bf27a242-c743-48e4-b757-fdd25f3af329 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.162798 Jul 20 15:28:27.646061 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2033/2960] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:27 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:27.945196 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:27.945794 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:27.952579 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:27.953024 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:27.953769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:27.954199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:28.091750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:28.100018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:28.101702 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:28.102858 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a3f5be7f-622e-464c-a1ad-5018c06bfd46 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.161659 Jul 20 15:28:28.103832 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2034/2961] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:27 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:28.680805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:28.681521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:28.688131 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:28.688576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:28.689340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:28.689772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:28.844466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:28.852206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-57e956f7-d4e7-42d4-95e1-9e588dcc3789 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.175957 Jul 20 15:28:28.852769 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2035/2962] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:28 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:29.122086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:29.123451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:29.129601 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:29.130045 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:29.130793 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:29.131266 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:29.273439 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:29.282115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:29.283687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:29.284844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-913de997-efdd-4722-8c18-03237078a873 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.167113 Jul 20 15:28:29.285461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2036/2963] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:29 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:29.872765 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:29.873442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:29.879706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:29.880158 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:29.880903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:29.881329 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:29.896208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:30.019187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:30.026763 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-32fb960b-6ceb-415d-8352-036720781e5c tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.158257 Jul 20 15:28:30.027364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2037/2964] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:29 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:30.303513 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:30.304121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:30.310602 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:30.311081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:30.311838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:30.312268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:30.447088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:30.455537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:30.457022 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:30.458177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-97db6425-a377-4b9b-872e-376949cd0b61 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.158686 Jul 20 15:28:30.458805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2038/2965] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:30 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:31.043264 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:31.044039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:31.050849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:31.051330 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:31.052073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:31.052481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:31.186161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:31.194396 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-765da02a-e634-440b-9049-1e0f7d3c737e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.155060 Jul 20 15:28:31.194967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2039/2966] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:31 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:31.479522 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:31.480129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:31.487036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:31.487520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:31.488327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:31.488758 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:31.636302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:31.645349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:31.646811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:31.648109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-626cfc4f-2724-4ebb-8026-e070d194747e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.172740 Jul 20 15:28:31.648942 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2040/2967] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:31 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:32.211610 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:32.212315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:32.218882 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:32.219365 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:32.220116 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:32.220553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:32.374516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:32.382237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ed981005-1bad-456c-b50c-d83e521f5e7e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.174473 Jul 20 15:28:32.383003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2041/2968] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:32 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:32.670209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:32.670844 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:32.677282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:32.677753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:32.678516 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:32.678947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:32.808756 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:32.816826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:32.818305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:32.819491 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e8d89d51-03ed-4516-aef2-92427c082615 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.153302 Jul 20 15:28:32.820030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2042/2969] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:32 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:33.399616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:33.400245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:33.406981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:33.407463 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:33.408307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:33.408627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:33.542791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:33.550582 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-78612876-8268-4ba6-be1c-fc983e7385bf tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.154809 Jul 20 15:28:33.551171 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2043/2970] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:33 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:33.837363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:33.837967 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:33.844525 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:33.844982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:33.845729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:33.846161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:33.977678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:33.986431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:33.987954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:33.989122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f742aa37-06e8-4d79-8e3d-c9329b78e289 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.155767 Jul 20 15:28:33.989680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2044/2971] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:33 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:34.568098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:34.568795 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:34.575543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:34.575988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:34.576732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:34.577157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:34.711544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:34.719305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a1e9f996-b946-413c-8d0c-2b71329ae054 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.155192 Jul 20 15:28:34.719874 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2045/2972] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:34 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:35.007481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:35.008088 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:35.014534 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:35.014982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:35.015779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:35.016213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:35.170219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:35.178921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:35.180456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:35.181595 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6aa28d07-4f13-4b9e-9c0e-60235e3007ee tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.178182 Jul 20 15:28:35.182192 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2046/2973] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:35 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:35.739198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:35.740019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:35.748754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:35.749197 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:35.749941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:35.750364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:35.882955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:35.891098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cb9e427f-81fd-4b10-a805-7dd4ffd365e4 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.157429 Jul 20 15:28:35.891727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2047/2974] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:35 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:36.203815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:36.204603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:36.212887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:36.213475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:36.214461 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:36.215027 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:36.622548 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:36.630686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:36.632198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:36.633355 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a3b6b704-5eca-48de-9bd1-372fe373d4b2 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.435579 Jul 20 15:28:36.633926 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2048/2975] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:36 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:36.911109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:36.912086 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:36.920142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:36.920572 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:36.921319 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:36.921750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:37.059114 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:37.067208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-47780ea7-1ded-4d02-9d50-2c48b688103b tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.161653 Jul 20 15:28:37.067770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2049/2976] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:36 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:37.660232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:37.661282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:37.670544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:37.670991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:37.671805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:37.672233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:37.819761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:37.830152 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:37.831687 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:37.832852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-111437db-fbb7-437f-a667-eb9a2074e836 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.179365 Jul 20 15:28:37.833421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2050/2977] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:37 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:38.088172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:38.089093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:38.096736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:38.097182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:38.097928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:38.098357 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:38.241144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:38.248912 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-40275456-1d7b-4033-addb-14df7b09c7c8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.167231 Jul 20 15:28:38.249537 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2051/2978] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:38 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:38.855267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:38.856655 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:38.866956 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:38.867762 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:38.869015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:38.870198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:39.022025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:39.031427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:39.037607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:39.038974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-f6249335-6a02-4676-a738-358a2acd4e4e tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.190286 Jul 20 15:28:39.039676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2052/2979] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:38 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:39.269940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:39.270992 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:39.283829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:39.284761 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:39.286759 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:39.287666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:39.423772 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:39.435798 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cd4bc501-d300-4415-8e98-0115a7960768 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.172389 Jul 20 15:28:39.436375 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2053/2980] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:39 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:40.061199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:40.062360 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:40.072254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:40.072997 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:40.074233 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:40.074961 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:40.077686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:40.213172 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:40.222755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:40.224283 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:40.225440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5bf3e42f-7abc-4ea6-b617-b960fc225c45 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.170858 Jul 20 15:28:40.226017 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2054/2981] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:40 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:40.456818 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:40.457887 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:40.467808 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:40.468186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:40.468941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:40.469368 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:40.610652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:40.618294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-24f9da66-e294-4f8c-98db-9070b7f7bdaa tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.167974 Jul 20 15:28:40.618873 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2055/2982] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:40 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:41.250242 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:41.250791 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:41.257364 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:41.257819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:41.258570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:41.259039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:41.401005 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:41.409536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:41.411010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:41.412189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-eff74a89-781e-44dd-8ac3-8f382aef369c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.166093 Jul 20 15:28:41.412753 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2056/2983] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:41 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:41.635726 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:41.636460 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:41.644612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:41.645250 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:41.646496 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:41.647316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:41.786063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:41.797166 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a612da86-b288-487d-8fdd-9c0362c47930 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.165283 Jul 20 15:28:41.797904 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2057/2984] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:41 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:42.430487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:42.431138 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:42.438308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:42.438750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:42.439570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:42.440007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:42.610037 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:42.618387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:42.619931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:42.621160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-73b1981b-8ca6-493f-a06f-dc4c0814f771 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.195023 Jul 20 15:28:42.621738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2058/2985] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:42 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:42.818880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:42.819716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:42.826470 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:42.826928 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:42.827723 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:42.828202 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:42.972215 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:42.983069 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-cfa5d3f8-97fc-42d5-bca4-fbd7e8c5c30e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.168108 Jul 20 15:28:42.983661 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2059/2986] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:42 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:43.639764 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:43.640642 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:43.646796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:43.647301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:43.648064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:43.648502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:43.787931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:43.795910 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:43.797409 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:43.798559 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a0923737-20fa-406c-bf39-7b8bf710513b tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.162865 Jul 20 15:28:43.799128 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2060/2987] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:43 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:44.000994 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:44.001676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:44.008618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:44.009159 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:44.009935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:44.010398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:44.165836 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:44.173381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-826800cd-3270-4b16-b8fe-38233669a65d tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1366 microversion: 2.1 time: 0.176512 Jul 20 15:28:44.173925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2061/2988] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:43 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1366 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:28:44.656165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-8efa5d71-1cd5-4f09-85d4-cd93b271aaf9 req-95ae715a-db32-4b09-a69a-252c4cdeafcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "name": "network-vif-plugged", "status": "completed", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:44.665122 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-8efa5d71-1cd5-4f09-85d4-cd93b271aaf9 req-95ae715a-db32-4b09-a69a-252c4cdeafcc service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:44.665777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-8efa5d71-1cd5-4f09-85d4-cd93b271aaf9 req-95ae715a-db32-4b09-a69a-252c4cdeafcc service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:44.678543 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-8efa5d71-1cd5-4f09-85d4-cd93b271aaf9 req-95ae715a-db32-4b09-a69a-252c4cdeafcc service nova] Creating event network-vif-plugged:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:28:44.684440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-8efa5d71-1cd5-4f09-85d4-cd93b271aaf9 req-95ae715a-db32-4b09-a69a-252c4cdeafcc service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033694 Jul 20 15:28:44.685169 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2062/2989] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:28:44.817498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:44.818178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:44.824562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:44.825308 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:44.826361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:44.826955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:44.993729 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:45.002155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:45.003821 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:45.005126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5bd32d57-bc84-4f76-8689-c624d9b7da2c tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.191794 Jul 20 15:28:45.005708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2063/2990] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:44 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:45.189536 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:45.190144 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:45.197130 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:45.197581 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:45.198343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:45.198775 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:45.348004 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:45.358861 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-65c8f145-30c2-4fc9-8bab-bfa732740116 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1554 microversion: 2.1 time: 0.173161 Jul 20 15:28:45.360278 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2064/2991] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:28:45 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1554 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:28:45.375363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:45.375954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:45.382562 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:45.383381 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:45.384142 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:45.384955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:45.513972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8d795238-13f5-4c36-b29a-ff53c1e01b2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.142234 Jul 20 15:28:45.514839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2065/2992] 10.222.0.15 () {58 vars in 1274 bytes} [Wed Jul 20 15:28:45 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-interface => generated 284 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:28:46.023153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:46.023886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:46.030586 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:46.031459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:46.032494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:46.033290 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:46.182737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:46.192165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:46.194012 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:46.195519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ea5d1960-08f2-48a9-adb3-d734d395afd5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.176361 Jul 20 15:28:46.199096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2066/2993] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:46 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:46.737433 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-0f076d77-ddcb-4d8b-b0f8-1ee612e97102 req-7cc16432-2f83-426b-8f70-6be7dce13220 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4464f0b3-f153-4575-9588-8bc7a46dadd5", "tag": "47f645c2-c472-46a5-ad91-9d245af7f214"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:28:46.747261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-0f076d77-ddcb-4d8b-b0f8-1ee612e97102 req-7cc16432-2f83-426b-8f70-6be7dce13220 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:46.747833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-0f076d77-ddcb-4d8b-b0f8-1ee612e97102 req-7cc16432-2f83-426b-8f70-6be7dce13220 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:46.764893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-0f076d77-ddcb-4d8b-b0f8-1ee612e97102 req-7cc16432-2f83-426b-8f70-6be7dce13220 service nova] Creating event network-changed:47f645c2-c472-46a5-ad91-9d245af7f214 for instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 on hv-cinder-84902 Jul 20 15:28:46.771686 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-0f076d77-ddcb-4d8b-b0f8-1ee612e97102 req-7cc16432-2f83-426b-8f70-6be7dce13220 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038229 Jul 20 15:28:46.772320 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2067/2994] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:28:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:28:47.216494 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:47.217921 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:47.226471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:47.226915 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:47.227880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:47.228311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:47.382362 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:47.390448 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:47.391953 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:47.393121 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5cf906a7-4dca-46e2-8e7d-a7017f15d0c4 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.182567 Jul 20 15:28:47.393678 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2068/2995] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:47 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:28:48.410828 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:48.411740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:48.415770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=100 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:28:48.420587 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:48.421157 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:48.422132 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:48.422663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:48.585711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:48.591254 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:48.593006 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:48.594300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ee89225f-595d-4a29-804c-d1efc2f92af5 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1732 microversion: 2.20 time: 0.187448 Jul 20 15:28:48.594962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2069/2996] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:48 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1732 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:28:49.612240 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:28:49.612846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:28:49.619829 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:49.620279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:49.621039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:28:49.621466 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:28:49.760337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:49.769860 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:49.772003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:28:49.773167 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e1af7b62-330e-4c10-bf14-e7108a1de987 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1719 microversion: 2.20 time: 0.164799 Jul 20 15:28:49.773728 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2070/2997] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:28:49 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1719 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:29:18.068274 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:29:18.069612 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:18.073178 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:18.074256 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=91 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:18.081524 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:18.081973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:18.082737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:18.083190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:18.158598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8bba127e-71c5-464b-9b3a-9f1e3a612478 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action" status: 202 len: 0 microversion: 2.1 time: 0.096920 Jul 20 15:29:18.159411 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2071/2998] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:29:18 2022] POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action => generated 0 bytes in 99 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:29:18.168036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:18.168738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:18.175292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:18.175750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:18.176618 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:18.177058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:18.420784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:18.431852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-009d060a-8355-42db-b293-96ca9f7c85c5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1749 microversion: 2.1 time: 0.267800 Jul 20 15:29:18.432673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2072/2999] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:18 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1749 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:29:19.453957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:19.455126 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:19.465917 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:19.466363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:19.467123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:19.467578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:19.624058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:19.632800 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a126b27b-c323-479b-a85d-25249df8a8c1 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.185316 Jul 20 15:29:19.633388 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2073/3000] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:19 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:20.653149 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:20.654146 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:20.664186 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:20.664810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:20.665968 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:20.666660 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:20.848301 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:20.861611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b05c4e8e-f9a3-4bb8-b5ac-09d1d75d338a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.213338 Jul 20 15:29:20.862784 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2074/3001] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:20 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:21.884541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:21.885421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:21.893153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:21.893627 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:21.894390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:21.894908 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:22.055382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:22.067954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c03891a8-866b-44a6-a198-4113431a243f tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.190519 Jul 20 15:29:22.068732 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2075/3002] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:21 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:23.090952 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:23.091733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:23.098424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:23.098870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:23.099666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:23.100093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:23.254611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:23.262771 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-672f2b62-5c76-41af-9c4d-58d361723a7a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.177646 Jul 20 15:29:23.263369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2076/3003] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:23 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:24.288770 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:24.289936 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:24.297932 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:24.298370 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:24.299139 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:24.299571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:24.451973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:24.460654 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-84397ce7-19b1-44be-9b47-a1e3aa5f5050 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.178282 Jul 20 15:29:24.461220 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2077/3004] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:24 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:25.480779 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:25.481553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:25.490590 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:25.491227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:25.492421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:25.493023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:25.649619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:25.658591 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-4cc5f29e-928a-4b8f-9e35-e5c7d0e24835 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.182960 Jul 20 15:29:25.659165 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2078/3005] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:25 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:26.680020 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:26.681123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:26.688737 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:26.689188 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:26.689939 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:26.690376 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:26.835906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:26.844629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ad37a80c-38da-4193-a5c5-16736c78e608 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.171032 Jul 20 15:29:26.845464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2079/3006] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:26 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:27.871505 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:27.872639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:27.884442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:27.885191 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:27.886471 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:27.886907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:28.039025 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:28.047247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-78589936-0c9f-45fc-8c7b-9eadac6bd088 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.182341 Jul 20 15:29:28.047815 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2080/3007] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:27 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:29.065055 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:29.065755 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:29.072385 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:29.072839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:29.073622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:29.074053 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:29.229993 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:29.242487 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b07e8a96-6f75-4ac1-8419-9ee7149316fb tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.181156 Jul 20 15:29:29.243314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2081/3008] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:29 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:30.264232 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:30.265084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:30.273180 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:30.273740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:30.274673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:30.275243 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:30.431847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:30.440440 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-c524f3ec-7561-4ed2-b3ad-9ae924df85b2 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.181841 Jul 20 15:29:30.441010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2082/3009] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:30 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:31.461991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:31.463091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:31.470481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:31.470929 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:31.471797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:31.472227 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:31.614216 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:31.622738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-433069bf-c21d-4ac0-860a-56f8c24cb51e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.167257 Jul 20 15:29:31.623348 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2083/3010] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:31 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:32.641720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:32.642639 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:32.652629 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:32.653289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:32.654349 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:32.654972 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:32.806346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:32.815340 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bd19ced3-d9e1-4e03-bbb5-2b52d355d11a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.179155 Jul 20 15:29:32.815919 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2084/3011] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:32 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:33.840668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:33.841811 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:33.852719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:33.853453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:33.855073 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:33.855564 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:34.000653 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:34.012310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-235e8b25-069d-4917-86c9-399d0a2a97d3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.178135 Jul 20 15:29:34.012862 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2085/3012] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:33 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:35.038796 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:35.040021 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:35.051007 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:35.051838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:35.053032 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:35.053740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:35.073727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:35.210356 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:35.220153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3abad83a-91e5-4b1b-b8c9-baf7ad8bcd89 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1754 microversion: 2.1 time: 0.187976 Jul 20 15:29:35.220727 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2086/3013] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:35 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1754 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:36.242258 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:36.243413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:36.255659 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:36.256426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:36.257749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:36.258565 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:36.406153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:36.423703 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-287c90d4-a0dc-4d33-b830-8321b6bd9b6d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1757 microversion: 2.1 time: 0.188304 Jul 20 15:29:36.424294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2087/3014] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:29:36 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1757 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:44.531082 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:29:44.531901 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:29:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:29:48.281020 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:29:48.281824 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:29:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:29:57.121739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:57.123174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:57.126985 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=34 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:57.127810 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=8 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:29:57.137124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.137566 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.138324 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.138750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.168265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fa4f2a22-2732-42c8-a1f0-9bdc60c045a1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.052243 Jul 20 15:29:57.168957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2088/3015] 10.222.0.15 () {60 vars in 1342 bytes} [Wed Jul 20 15:29:57 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:29:57.227924 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:57.228416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:57.235351 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.235789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.236533 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.236957 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.262292 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:29:57.345699 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6848bffb-0763-4075-8793-6ad320e74cd8 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 204 len: 0 microversion: 2.20 time: 0.121199 Jul 20 15:29:57.346223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2089/3016] 10.222.0.15 () {60 vars in 1279 bytes} [Wed Jul 20 15:29:57 2022] DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 20 15:29:57.355680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:57.356363 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:57.363074 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.363532 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.364289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:57.364711 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:57.500300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:57.508735 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:57.510241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:57.511437 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-fd25e62f-8ddc-4c4d-8618-2c17de46a366 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.160299 Jul 20 15:29:57.511995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2090/3017] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:29:57 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 20 15:29:58.530147 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:58.530851 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:58.537773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:58.538212 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:58.538954 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:58.539444 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:58.676619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:58.688833 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:58.691261 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:58.693123 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-11e85dc3-811a-4f25-8879-0abd9efc9a63 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.166713 Jul 20 15:29:58.693979 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2091/3018] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:29:58 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:29:59.712736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:29:59.713651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:29:59.720616 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:59.721065 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:59.721839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:29:59.722271 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:29:59.955509 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:59.964187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:59.965675 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ee5a234904e64eb9b2faad85909d0507', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7db1a5ab36714ee2990b9c6c10ba0aea', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:29:59.966845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d3c21a39-ae5e-4674-9efc-45fa1ab0dd94 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 200 len: 1725 microversion: 2.20 time: 0.258221 Jul 20 15:29:59.967458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2092/3019] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:29:59 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 1725 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 20 15:30:00.183420 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-be439412-54d7-456a-9aaf-34ca7cc4110d req-10820b56-b306-44e7-9870-76954e50bacd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "70521d7d-d55b-40ff-a8b8-8579dcba3c98", "name": "network-vif-deleted", "tag": "3fa39ece-8a04-4c90-88e2-aac70fecbbc4"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:00.193311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-be439412-54d7-456a-9aaf-34ca7cc4110d req-10820b56-b306-44e7-9870-76954e50bacd service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:00.194016 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-be439412-54d7-456a-9aaf-34ca7cc4110d req-10820b56-b306-44e7-9870-76954e50bacd service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:00.207315 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-be439412-54d7-456a-9aaf-34ca7cc4110d req-10820b56-b306-44e7-9870-76954e50bacd service nova] Creating event network-vif-deleted:3fa39ece-8a04-4c90-88e2-aac70fecbbc4 for instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 on hv-cinder-84902 Jul 20 15:30:00.212237 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-be439412-54d7-456a-9aaf-34ca7cc4110d req-10820b56-b306-44e7-9870-76954e50bacd service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032576 Jul 20 15:30:00.216434 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2093/3020] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:30:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:30:00.992877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:00.994720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:01.009570 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:01.010161 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:01.011331 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:01.011846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:01.026733 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 HTTP/1.1 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:01.028424 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:01.030475 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '70521d7d-d55b-40ff-a8b8-8579dcba3c98'} Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:01.031663 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:01.033310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:01.035040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:01.035040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-dd275b09-73e9-4d45-8a31-0830a973318f tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 404 len: 111 microversion: 2.20 time: 0.047773 Jul 20 15:30:01.035040 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2094/3021] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:30:00 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:30:01.039459 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-25b9da88-7c29-43f5-b72c-7b593c5a9178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '.version_select at 0x7f7b15bd78b0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:01.061974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-25b9da88-7c29-43f5-b72c-7b593c5a9178 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2021460468" status: 204 len: 0 microversion: 2.20 time: 0.026155 Jul 20 15:30:01.062842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2095/3022] 10.222.0.15 () {60 vars in 1261 bytes} [Wed Jul 20 15:30:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2021460468 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 20 15:30:02.916742 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:02.917395 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:02.924598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:02.925058 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:02.925819 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:02.926230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:02.940070 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 HTTP/1.1 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:02.941673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.943479 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '70521d7d-d55b-40ff-a8b8-8579dcba3c98'} Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:02.944443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.946160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.947973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:02.947973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-49e08f30-33c4-4c4d-bd90-fa92796ee89a tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 404 len: 111 microversion: 2.20 time: 0.030791 Jul 20 15:30:02.947973 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2096/3023] 10.222.0.15 () {60 vars in 1279 bytes} [Wed Jul 20 15:30:02 2022] DELETE /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:30:02.951656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:02.952279 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 70521d7d-d55b-40ff-a8b8-8579dcba3c98] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:02.958576 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:02.959026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:02.959807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:02.960241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:02.974248 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 HTTP/1.1 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:02.975799 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.977641 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '70521d7d-d55b-40ff-a8b8-8579dcba3c98'} Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:02.978959 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. Jul 20 15:30:02.980707 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:02.982743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 70521d7d-d55b-40ff-a8b8-8579dcba3c98 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:02.982743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-5b317165-b8cb-42d3-89bd-dd459f92fc72 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98" status: 404 len: 111 microversion: 2.20 time: 0.029162 Jul 20 15:30:02.982743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2097/3024] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:30:02 2022] GET /compute/v2.1/servers/70521d7d-d55b-40ff-a8b8-8579dcba3c98 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 20 15:30:04.143182 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:04.143807 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:04.150495 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:04.150935 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:04.151716 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:04.152141 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:04.167781 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d HTTP/1.1 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:04.169431 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.171521 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d'} Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:04.172603 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.174337 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.176103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:04.176103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-34c514ce-e1fb-47f2-ba6a-3af8117179f1 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 404 len: 111 microversion: 2.20 time: 0.031580 Jul 20 15:30:04.176103 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2098/3025] 10.222.0.15 () {60 vars in 1279 bytes} [Wed Jul 20 15:30:04 2022] DELETE /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:30:04.179081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:04.179720 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] [instance: 3dfe0713-c262-4daa-8e0d-d439f6e68f7d] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:04.185867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:04.186310 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:04.187071 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:04.187553 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:04.200740 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] HTTP exception thrown: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Request method failure captured: Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d HTTP/1.1 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWjAsiUtfpVTlZ63FUwbO4bAAzrMZCGSl2f5SgKE7D0aCLzlrG-qz2MSIrkjhdp3QYr8wBVTlBCJ7hTgx3SXbZlgqSQI_4TWTyowY0mIzJw4H9e0Qp0WbdOU6IoR8d4REXp6Hxz-_QBBoPN2ObOW5Le7ki9XqlS_LTScSmoASox_pI Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Openstack-Nova-Api-Version: 2.20 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:04.202219 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/7db1a5ab36714ee2990b9c6c10ba0aea"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/7db1a5ab36714ee2990b9c6c10ba0aea"}]}] Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 7db1a5ab36714ee2990b9c6c10ba0aea Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-241591026 Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: ee5a234904e64eb9b2faad85909d0507 Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeShelveTestJSON-241591026-project Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.204611 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '3dfe0713-c262-4daa-8e0d-d439f6e68f7d'} Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:04.205845 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. Jul 20 15:30:04.207712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:04.209295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] Returning 404 to user: Instance 3dfe0713-c262-4daa-8e0d-d439f6e68f7d could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:04.209295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e127a8a1-0b0f-417f-90d7-cea7cbadbe95 tempest-AttachVolumeShelveTestJSON-241591026 tempest-AttachVolumeShelveTestJSON-241591026-project] 10.222.0.15 "GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d" status: 404 len: 111 microversion: 2.20 time: 0.028122 Jul 20 15:30:04.209295 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2099/3026] 10.222.0.15 () {60 vars in 1276 bytes} [Wed Jul 20 15:30:04 2022] GET /compute/v2.1/servers/3dfe0713-c262-4daa-8e0d-d439f6e68f7d => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 20 15:30:08.828960 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:30:08.829588 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:30:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:30:08.864749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-776640a0-cc41-4e3c-9483-c9519793838e tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Action: 'create', calling method: .version_select at 0x7f7b15b66700>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1655062515", "metadata": {"Type": "work"}}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:09.190958 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-776640a0-cc41-4e3c-9483-c9519793838e tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.465026 Jul 20 15:30:09.191891 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2100/3027] 10.222.0.15 () {60 vars in 1155 bytes} [Wed Jul 20 15:30:08 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 467 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jul 20 15:30:09.220451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1523240d-942d-4e45-b2a5-0b92c9c0e46c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b16002160>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:09.295852 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1523240d-942d-4e45-b2a5-0b92c9c0e46c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 200 len: 353 microversion: 2.1 time: 0.093486 Jul 20 15:30:09.296447 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2101/3028] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 15:30:09 2022] GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 353 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:10.311300 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-17cd0fa8-3dda-4b5e-9cfa-7e46d6276b0e tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b15bd7820>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:10.367323 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-17cd0fa8-3dda-4b5e-9cfa-7e46d6276b0e tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 200 len: 354 microversion: 2.1 time: 0.061832 Jul 20 15:30:10.367886 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2102/3029] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 15:30:10 2022] GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 354 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:10.378405 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-1c24b33a-a87f-4b53-8b60-b7fbea942f95 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b15bee1f0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:10.426206 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-1c24b33a-a87f-4b53-8b60-b7fbea942f95 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 200 len: 354 microversion: 2.1 time: 0.051033 Jul 20 15:30:10.426955 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2103/3030] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 15:30:10 2022] GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 354 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:10.446962 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-931d9b20-7e77-4ee0-b203-944bf207de01 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b15be6700>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:10.515369 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-931d9b20-7e77-4ee0-b203-944bf207de01 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "DELETE /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 202 len: 0 microversion: 2.1 time: 0.075469 Jul 20 15:30:10.515940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2104/3031] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 15:30:10 2022] DELETE /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:30:10.523826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-680bb4f7-677b-463e-8787-6b09bf2ce6f6 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b15cfb430>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:10.579018 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-680bb4f7-677b-463e-8787-6b09bf2ce6f6 tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 200 len: 353 microversion: 2.1 time: 0.058676 Jul 20 15:30:10.579656 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2105/3032] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 15:30:10 2022] GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 353 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:11.596681 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebdc8c08-2a41-4ae7-b7e7-0d36f891bc4c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Calling method '.version_select at 0x7f7b15dc6ee0>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:11.628030 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-ebdc8c08-2a41-4ae7-b7e7-0d36f891bc4c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] HTTP exception thrown: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebdc8c08-2a41-4ae7-b7e7-0d36f891bc4c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Request method failure captured: Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa HTTP/1.1 Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2B-APfe-6YEs_XGtPBZp5DYfafZ79kUV50i4O3UKLLtjRNV8uA4fcJacXxYyHidQgJjJ327n6txfOebETOjQmjIcFyglRWIaog8tNwTKeLJE2_VQUgmrEx71hFazy3lw2n54FNRGXOgiPWO1qxualbbfwvCtR6nouNp7HjtqYJTk11c Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: a2891c3f9d95446b951eb7abe7624f6d Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-VolumesGetTestJSON-1637095430 Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:11.629978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_a2891c3f9d95446b951eb7abe7624f6d"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/a2891c3f9d95446b951eb7abe7624f6d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/a2891c3f9d95446b951eb7abe7624f6d"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/a2891c3f9d95446b951eb7abe7624f6d"}]}] Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-VolumesGetTestJSON-1637095430 Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: a2891c3f9d95446b951eb7abe7624f6d Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-VolumesGetTestJSON-1637095430 Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-VolumesGetTestJSON-1637095430-project Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: c5516a90c978481593de115b69a42a2b Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-VolumesGetTestJSON-1637095430-project Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'b8b9e207-6d52-4461-845e-cea80344d2fa'} Jul 20 15:30:11.631847 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. (HTTP 404) (Request-ID: req-b0a4882d-50ed-4a35-b3c5-d73a6266fff7) Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:11.633402 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 20 15:30:11.635528 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 20 15:30:11.637305 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ebdc8c08-2a41-4ae7-b7e7-0d36f891bc4c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] Returning 404 to user: Volume b8b9e207-6d52-4461-845e-cea80344d2fa could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ebdc8c08-2a41-4ae7-b7e7-0d36f891bc4c tempest-VolumesGetTestJSON-1637095430 tempest-VolumesGetTestJSON-1637095430-project] 10.222.0.15 "GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa" status: 404 len: 109 microversion: 2.1 time: 0.042263 Jul 20 15:30:11.639647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2106/3033] 10.222.0.15 () {58 vars in 1244 bytes} [Wed Jul 20 15:30:11 2022] GET /compute/v2.1/os-volumes/b8b9e207-6d52-4461-845e-cea80344d2fa => generated 109 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:16.742649 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:16.743442 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:16.940498 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:16.941031 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:16.947743 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:16.948195 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:16.948978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:16.949408 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:16.977209 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9de3c79f-858c-4420-9350-d8d5e34dc4fe tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/ips" status: 200 len: 148 microversion: 2.1 time: 0.235857 Jul 20 15:30:16.977855 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2107/3034] 10.222.0.15 () {58 vars in 1247 bytes} [Wed Jul 20 15:30:16 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/ips => generated 148 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:23.589950 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "374f4b62-d367-4e56-b420-560dd6820a8f"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:23.590960 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:23.597387 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:23.597846 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:23.598619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:23.599666 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:23.910133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d1d5034f-de5d-4513-b205-e79a85209be6 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "POST /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.324022 Jul 20 15:30:23.910776 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2108/3035] 10.222.0.15 () {60 vars in 1322 bytes} [Wed Jul 20 15:30:23 2022] POST /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments => generated 194 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:30:31.565081 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:31.565749 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:31.572316 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:31.572805 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:31.573619 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:31.574050 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:31.669416 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'services' on Instance uuid 4464f0b3-f153-4575-9588-8bc7a46dadd5 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:30:31.750925 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-31ec274b-ff8c-4e6b-ab3e-26baab94a603 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments/374f4b62-d367-4e56-b420-560dd6820a8f" status: 202 len: 0 microversion: 2.1 time: 0.190429 Jul 20 15:30:31.751652 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2109/3036] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:30:31 2022] DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments/374f4b62-d367-4e56-b420-560dd6820a8f => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:30:31.762064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:31.763235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:31.770502 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:31.771003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:31.771867 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:31.772304 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:31.806311 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-17d2b161-2388-4294-ac1f-0c7e429d3ddd tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048550 Jul 20 15:30:31.806911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2110/3037] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:30:31 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:32.825469 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:32.826550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:32.837014 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:32.840651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:32.840651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:32.840651 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:32.897913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a9586f59-9ce8-4d5b-960c-74a136db3196 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.081006 Jul 20 15:30:32.899124 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2111/3038] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:30:32 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5/os-volume_attachments => generated 25 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 20 15:30:32.969117 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:32.969748 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:32.977160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:32.977854 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:32.978712 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:32.979177 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:33.018679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:30:33.117333 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-003dbd1d-e644-447b-91eb-a42d818ce9e8 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 204 len: 0 microversion: 2.1 time: 0.152768 Jul 20 15:30:33.117948 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2112/3039] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:32 2022] DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 20 15:30:33.128485 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:33.128998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:33.135093 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:33.135549 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:33.136452 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:33.136739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:33.279614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:33.288569 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-2d7b5231-6aa5-46f8-9667-ad2486046771 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1679 microversion: 2.1 time: 0.165778 Jul 20 15:30:33.289190 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2113/3040] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:33 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1679 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:30:34.318153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:34.318885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:34.327920 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:34.328679 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:34.329480 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:34.329911 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:34.497622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:34.509477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6fe32d24-75c9-4a01-84b9-e525663bab5e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1679 microversion: 2.1 time: 0.196381 Jul 20 15:30:34.510068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2114/3041] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:34 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1679 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:30:35.528598 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:35.529519 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:35.538988 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:35.539462 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:35.540270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:35.540864 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:35.766773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:35.783028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-92f8486a-95ba-4d23-a2ee-338424a45713 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1679 microversion: 2.1 time: 0.254198 Jul 20 15:30:35.783028 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2115/3042] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:35 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1679 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:30:36.612072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-3bca1bca-304a-4b28-bdde-0b3792fb29fe req-7542c0ee-7f6d-4a82-b991-b20d6d9023d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4464f0b3-f153-4575-9588-8bc7a46dadd5", "name": "network-vif-deleted", "tag": "47f645c2-c472-46a5-ad91-9d245af7f214"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:36.623571 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3bca1bca-304a-4b28-bdde-0b3792fb29fe req-7542c0ee-7f6d-4a82-b991-b20d6d9023d9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:36.624343 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-3bca1bca-304a-4b28-bdde-0b3792fb29fe req-7542c0ee-7f6d-4a82-b991-b20d6d9023d9 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:36.644490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-3bca1bca-304a-4b28-bdde-0b3792fb29fe req-7542c0ee-7f6d-4a82-b991-b20d6d9023d9 service nova] Creating event network-vif-deleted:47f645c2-c472-46a5-ad91-9d245af7f214 for instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 on hv-cinder-84902 Jul 20 15:30:36.653398 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-3bca1bca-304a-4b28-bdde-0b3792fb29fe req-7542c0ee-7f6d-4a82-b991-b20d6d9023d9 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044881 Jul 20 15:30:36.654945 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2116/3043] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:30:36.808550 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:36.809575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:36.816980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:36.817922 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:36.819451 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:36.820044 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:36.864427 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:36.865153 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:36.865777 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lazy-loading 'fault' on Instance uuid 4464f0b3-f153-4575-9588-8bc7a46dadd5 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:30:36.920621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e3574b5b8f3241b192b0b810122d4769', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5d984345e0d340818c9030f70bace6d3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:36.928449 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ce7055c8-8e2c-483a-882b-bd6b91d41109 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 200 len: 1330 microversion: 2.1 time: 0.125793 Jul 20 15:30:36.929393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2117/3044] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:36 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 1330 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 20 15:30:37.946099 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:37.947535 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:37.955173 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:37.955646 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:37.956421 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:37.956849 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:37.975094 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 HTTP/1.1 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:37.976739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:37.979647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4464f0b3-f153-4575-9588-8bc7a46dadd5'} Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:37.981280 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:37.984205 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:37.986746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:37.986746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8735559b-dee5-4682-8b6e-27ab5223e218 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 404 len: 111 microversion: 2.1 time: 0.036340 Jul 20 15:30:37.986746 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2118/3045] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:37 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:37.992906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-85aa3d18-5878-41ea-9027-3442ec9ecc68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '.version_select at 0x7f7b15cc7c10>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:38.013964 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-85aa3d18-5878-41ea-9027-3442ec9ecc68 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-26907741" status: 202 len: 0 microversion: 2.1 time: 0.024952 Jul 20 15:30:38.014696 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2119/3046] 10.222.0.15 () {58 vars in 1214 bytes} [Wed Jul 20 15:30:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-26907741 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:30:39.967174 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:39.967754 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:39.978213 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:39.978744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:39.979668 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:39.980105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:39.995674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 HTTP/1.1 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:40.001366 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.003358 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4464f0b3-f153-4575-9588-8bc7a46dadd5'} Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:40.004738 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.007273 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.010189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:40.010189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-20acaebd-e01d-4916-ab0f-43c71942748a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 404 len: 111 microversion: 2.1 time: 0.046664 Jul 20 15:30:40.010189 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2120/3047] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:39 2022] DELETE /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:30:40.016399 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:40.017010 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 4464f0b3-f153-4575-9588-8bc7a46dadd5] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:40.024593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:40.025285 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:40.026155 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:40.026674 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:40.047870 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 HTTP/1.1 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:40.050450 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.052230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '4464f0b3-f153-4575-9588-8bc7a46dadd5'} Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:40.053361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. Jul 20 15:30:40.056499 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:40.058673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 4464f0b3-f153-4575-9588-8bc7a46dadd5 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:40.058673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-b54680df-610e-42b5-8525-5aea8712676a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5" status: 404 len: 111 microversion: 2.1 time: 0.039898 Jul 20 15:30:40.058673 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2121/3048] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:40 2022] GET /compute/v2.1/servers/4464f0b3-f153-4575-9588-8bc7a46dadd5 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:41.209085 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:41.209671 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:41.216701 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.217185 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.217906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.218332 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.232540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 HTTP/1.1 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:41.234096 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.235241 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'fbbcd35f-cd1f-4195-958f-1e9d1194aea0'} Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:41.235866 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.237129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.238198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:41.238198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3bb2b269-3d94-421e-8f11-1141e2316671 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 404 len: 111 microversion: 2.1 time: 0.030469 Jul 20 15:30:41.238198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2122/3049] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:41 2022] DELETE /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:41.243906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:41.244507 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: fbbcd35f-cd1f-4195-958f-1e9d1194aea0] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:41.250544 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.250983 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.251750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.252200 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.265941 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 HTTP/1.1 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:41.267477 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.268578 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': 'fbbcd35f-cd1f-4195-958f-1e9d1194aea0'} Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:41.269293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. Jul 20 15:30:41.271110 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.272978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance fbbcd35f-cd1f-4195-958f-1e9d1194aea0 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:41.272978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-e0738ee7-d2a4-450c-8a49-8c4526e2640a tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0" status: 404 len: 111 microversion: 2.1 time: 0.028628 Jul 20 15:30:41.272978 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2123/3050] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:41 2022] GET /compute/v2.1/servers/fbbcd35f-cd1f-4195-958f-1e9d1194aea0 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:30:41.280393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:41.280885 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:41.287247 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.287706 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.288478 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.288901 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.303443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 HTTP/1.1 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:41.304981 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.306406 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '88eb690a-bcac-4a8e-82df-86f0d0280558'} Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:41.306995 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.308378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.309510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:41.309510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-ca572756-590b-4f1f-b41d-fae4160a0608 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 404 len: 111 microversion: 2.1 time: 0.030960 Jul 20 15:30:41.309510 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2124/3051] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:41 2022] DELETE /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:41.314692 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:41.315336 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 88eb690a-bcac-4a8e-82df-86f0d0280558] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:41.321974 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.322458 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.323282 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:41.323730 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:41.338695 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 HTTP/1.1 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:41.340426 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.341657 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '88eb690a-bcac-4a8e-82df-86f0d0280558'} Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:41.342327 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. Jul 20 15:30:41.343931 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:41.344906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 88eb690a-bcac-4a8e-82df-86f0d0280558 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:41.344906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-915ae8a8-2834-4f53-96bb-e14afa6c7205 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558" status: 404 len: 111 microversion: 2.1 time: 0.030830 Jul 20 15:30:41.344906 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2125/3052] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:41 2022] GET /compute/v2.1/servers/88eb690a-bcac-4a8e-82df-86f0d0280558 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:42.488268 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:42.488804 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:42.495539 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:42.495987 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:42.496736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:42.497160 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:42.501163 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:42.512089 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 HTTP/1.1 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:42.513839 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.514893 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '6b566c6c-a418-4384-b99e-c7295e754e92'} Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:42.515520 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.516546 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.517589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:42.517589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-a4ddc3a2-c1ee-4dd3-a70e-e149dca940e5 tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 404 len: 111 microversion: 2.1 time: 0.030972 Jul 20 15:30:42.517589 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2126/3053] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:42 2022] DELETE /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:42.523694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:42.524293 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] [instance: 6b566c6c-a418-4384-b99e-c7295e754e92] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:42.531270 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:42.531710 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:42.532467 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:42.532899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:42.546551 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] HTTP exception thrown: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Request method failure captured: Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 HTTP/1.1 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2BWioNknH0Ric25vwDEFARyMxkHN1Pac8wf4ECdtgTaNRMDm2Q1pkkrq9kEgItZT8___k24v1H4fraU2ds-EoZ5_X89pBszyL2pFfMxqNWtqoy0CaxykZfd3D0M6XG9ElQUCXHesqG0_q9ktpR1WkW7t-hyYL-Z8M8JJ6c0VzVk60Y0 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:42.548092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_5d984345e0d340818c9030f70bace6d3"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/5d984345e0d340818c9030f70bace6d3"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/5d984345e0d340818c9030f70bace6d3"}]}] Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 5d984345e0d340818c9030f70bace6d3 Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1584252508 Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: e3574b5b8f3241b192b0b810122d4769 Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-AttachVolumeNegativeTest-1584252508-project Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.550059 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '6b566c6c-a418-4384-b99e-c7295e754e92'} Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:42.551307 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. Jul 20 15:30:42.553391 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:42.555359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] Returning 404 to user: Instance 6b566c6c-a418-4384-b99e-c7295e754e92 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:42.555359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-8b25707a-6d6c-4fe0-8fcd-fc1a5a31ef2e tempest-AttachVolumeNegativeTest-1584252508 tempest-AttachVolumeNegativeTest-1584252508-project] 10.222.0.15 "GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92" status: 404 len: 111 microversion: 2.1 time: 0.029624 Jul 20 15:30:42.555359 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2127/3054] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:42 2022] GET /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:30:48.059068 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1127816965"}} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:48.060481 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:48.063430 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG dbcounter [-] [124396] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=124396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 15:30:48.070063 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:48.070789 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:48.072038 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:48.072302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:48.226515 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.network.security_group_api [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Removing security group 0245e78b-f532-40e7-a1a5-52b09cac1c15 from port bcb4295e-f687-475b-a18b-9de00faa1668 Jul 20 15:30:48.709245 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9583b46d-6921-49e4-94a1-0bd08e6f447a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action" status: 202 len: 0 microversion: 2.1 time: 0.655001 Jul 20 15:30:48.709857 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2128/3055] 10.222.0.15 () {60 vars in 1277 bytes} [Wed Jul 20 15:30:48 2022] POST /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/action => generated 0 bytes in 657 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 20 15:30:48.722991 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-9cfc59b9-ddb7-45d0-b365-2594ef0be052 req-d6e8da83-19e9-42d1-888d-c5ffb952e760 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:48.734636 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9cfc59b9-ddb7-45d0-b365-2594ef0be052 req-d6e8da83-19e9-42d1-888d-c5ffb952e760 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:48.735084 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9cfc59b9-ddb7-45d0-b365-2594ef0be052 req-d6e8da83-19e9-42d1-888d-c5ffb952e760 service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:48.760980 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-9cfc59b9-ddb7-45d0-b365-2594ef0be052 req-d6e8da83-19e9-42d1-888d-c5ffb952e760 service nova] Creating event network-changed:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:30:48.775039 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-9cfc59b9-ddb7-45d0-b365-2594ef0be052 req-d6e8da83-19e9-42d1-888d-c5ffb952e760 service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055255 Jul 20 15:30:48.776019 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2129/3056] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:30:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:30:50.650309 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:50.651015 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:50.660736 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:50.661194 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:50.662076 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:50.662512 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:50.771880 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.objects.instance [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lazy-loading 'services' on Instance uuid 95f1b8a6-b4c5-484f-8106-49c1aab6d333 {{(pid=124396) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 20 15:30:50.886382 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-72decb03-37e2-4724-9755-27546c41069e tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments/a81836ca-bd09-4ad6-8476-3e383ed3eea6" status: 202 len: 0 microversion: 2.1 time: 0.242914 Jul 20 15:30:50.887214 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2130/3057] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:30:50 2022] DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments/a81836ca-bd09-4ad6-8476-3e383ed3eea6 => generated 0 bytes in 245 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:30:50.907026 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-9c26a217-ad1f-4b03-84cd-12daa0cec6c7 req-7fbf6556-7fe7-4e8e-8537-72df6fe299fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:50.924622 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9c26a217-ad1f-4b03-84cd-12daa0cec6c7 req-7fbf6556-7fe7-4e8e-8537-72df6fe299fb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:50.926792 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-9c26a217-ad1f-4b03-84cd-12daa0cec6c7 req-7fbf6556-7fe7-4e8e-8537-72df6fe299fb service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:50.947745 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-9c26a217-ad1f-4b03-84cd-12daa0cec6c7 req-7fbf6556-7fe7-4e8e-8537-72df6fe299fb service nova] Creating event network-changed:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:30:50.955594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-9c26a217-ad1f-4b03-84cd-12daa0cec6c7 req-7fbf6556-7fe7-4e8e-8537-72df6fe299fb service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066410 Jul 20 15:30:50.957286 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2131/3058] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:30:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:50.968838 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:50.970744 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:50.978353 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:50.979235 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:50.980187 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:50.980826 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:51.050023 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-31c32176-9e10-4fd0-b837-3cb1533f5a46 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086179 Jul 20 15:30:51.057647 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2132/3059] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:30:50 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:52.062198 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:52.063456 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:52.071517 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:52.072091 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:52.073097 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:52.073540 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:52.104294 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-de484600-8d2d-4f80-8e85-b3cf63146e7b tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046023 Jul 20 15:30:52.105067 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2133/3060] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:30:52 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 20 15:30:53.116393 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:53.117432 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:53.124541 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:53.124996 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:53.125760 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:53.126201 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:53.155446 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-d15e0650-1e22-4a77-9115-466cecf505ae tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043023 Jul 20 15:30:53.156036 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2134/3061] 10.222.0.15 () {58 vars in 1301 bytes} [Wed Jul 20 15:30:53 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 20 15:30:53.166468 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:53.166965 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:53.173739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:53.174183 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:53.174938 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:53.175435 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:53.252739 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] HTTP exception thrown: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 is not attached to volume a81836ca-bd09-4ad6-8476-3e383ed3eea6 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Request method failure captured: Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments/a81836ca-bd09-4ad6-8476-3e383ed3eea6 HTTP/1.1 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2B6rXjv7m7Ntp7tQEwk6U8xHxSUi-V-VzuzVvJ5a1VyGyCESPP7K3XvRl_8zBNbrbs49kYCbbyOfV2f8gxTx1iDaSvc7AgCGjnYkf1RJt7kAPuo6b1wlLk14dFVgm_MIQBG5fnCuCHBs2h0oxiLQ2N9FbnFVd7wNBUT-m0KoStXkMPY Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 535d117f782341d5aa579a8fb34c35c5 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:53.263133 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/535d117f782341d5aa579a8fb34c35c5"}]}] Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 535d117f782341d5aa579a8fb34c35c5 Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestMinimumBasicScenario-1019241803-project Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 9b70cbf2ad4f405b84cbea240b83cd05 Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestMinimumBasicScenario-1019241803-project Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:53.264500 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 is not attached to volume a81836ca-bd09-4ad6-8476-3e383ed3eea6 Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'server_id': '95f1b8a6-b4c5-484f-8106-49c1aab6d333', 'id': 'a81836ca-bd09-4ad6-8476-3e383ed3eea6'} Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id a81836ca-bd09-4ad6-8476-3e383ed3eea6. Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:53.265064 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 is not attached to volume a81836ca-bd09-4ad6-8476-3e383ed3eea6 Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 is not attached to volume a81836ca-bd09-4ad6-8476-3e383ed3eea6 Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:53.266199 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Returning 404 to user: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 is not attached to volume a81836ca-bd09-4ad6-8476-3e383ed3eea6 {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:53.267204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7430a008-4b33-4ab6-995b-b47c0a7115b5 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments/a81836ca-bd09-4ad6-8476-3e383ed3eea6" status: 404 len: 154 microversion: 2.1 time: 0.102151 Jul 20 15:30:53.267204 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2135/3062] 10.222.0.15 () {58 vars in 1415 bytes} [Wed Jul 20 15:30:53 2022] DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333/os-volume_attachments/a81836ca-bd09-4ad6-8476-3e383ed3eea6 => generated 154 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 20 15:30:54.421092 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:54.421670 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:54.430842 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:54.431378 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:54.432154 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:54.432593 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:54.459483 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Going to try to terminate instance {{(pid=124396) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 20 15:30:54.564614 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-3fdf3db6-5de2-45cf-9049-dc9652bb985a tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 204 len: 0 microversion: 2.1 time: 0.147166 Jul 20 15:30:54.565750 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2136/3063] 10.222.0.15 () {58 vars in 1238 bytes} [Wed Jul 20 15:30:54 2022] DELETE /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 20 15:30:54.574453 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:54.575390 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:54.582413 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:54.582773 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:54.583607 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:54.584817 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:54.751574 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:54.759314 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-7e5c0038-63a6-4a5f-9ecc-2790b53a33e3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1552 microversion: 2.1 time: 0.189124 Jul 20 15:30:54.759877 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2137/3064] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:54 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1552 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:30:55.777231 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:55.777982 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:55.785223 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:55.785694 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:55.786464 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:55.786899 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:56.250575 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:56.279708 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-6de9cd88-4506-4bf7-b4a2-ca3111d2a944 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1552 microversion: 2.1 time: 0.506456 Jul 20 15:30:56.280306 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2138/3065] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:55 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1552 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:30:57.297289 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:57.297907 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:57.306003 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:57.306903 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:57.308115 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:57.309034 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:57.564719 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.policy [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b70cbf2ad4f405b84cbea240b83cd05', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '535d117f782341d5aa579a8fb34c35c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=124396) authorize /opt/stack/nova/nova/policy.py:199}} Jul 20 15:30:57.574267 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bad37fdd-3ccc-4ff6-a42f-e109fc41ca8d tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 200 len: 1552 microversion: 2.1 time: 0.280761 Jul 20 15:30:57.575637 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2139/3066] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:57 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 1552 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 20 15:30:57.819797 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [req-7dd61145-e9ff-474e-baff-357d998933b2 req-60f3f2ba-7bea-4b16-99c3-6f9f777d689e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "95f1b8a6-b4c5-484f-8106-49c1aab6d333", "name": "network-vif-deleted", "tag": "bcb4295e-f687-475b-a18b-9de00faa1668"}]} {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 20 15:30:57.836361 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7dd61145-e9ff-474e-baff-357d998933b2 req-60f3f2ba-7bea-4b16-99c3-6f9f777d689e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:57.837072 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [req-7dd61145-e9ff-474e-baff-357d998933b2 req-60f3f2ba-7bea-4b16-99c3-6f9f777d689e service nova] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:57.860265 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.compute.server_external_events [req-7dd61145-e9ff-474e-baff-357d998933b2 req-60f3f2ba-7bea-4b16-99c3-6f9f777d689e service nova] Creating event network-vif-deleted:bcb4295e-f687-475b-a18b-9de00faa1668 for instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 on hv-cinder-84902 Jul 20 15:30:57.869107 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [req-7dd61145-e9ff-474e-baff-357d998933b2 req-60f3f2ba-7bea-4b16-99c3-6f9f777d689e service nova] 10.222.0.15 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052994 Jul 20 15:30:57.870230 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2140/3067] 10.222.0.15 () {64 vars in 1317 bytes} [Wed Jul 20 15:30:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 20 15:30:58.598098 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:58.599816 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.compute.api [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] [instance: 95f1b8a6-b4c5-484f-8106-49c1aab6d333] Fetching instance by UUID {{(pid=124396) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 20 15:30:58.607443 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:58.607894 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:58.608676 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 20 15:30:58.609129 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG oslo_concurrency.lockutils [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Lock "f84c0e38-8de4-44d7-ae90-44f10e366750" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=124396) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 20 15:30:58.622594 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.wsgi [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] HTTP exception thrown: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Request method failure captured: Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: request: GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 HTTP/1.1 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept: application/json Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Accept-Encoding: identity Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Connection: close Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Length: 0 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Content-Type: application/json Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Host: 10.222.0.15 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Openstack-System-Scope: None Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: User-Agent: python-urllib3/1.26.9 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Auth-Token: gAAAAABi2B6rXjv7m7Ntp7tQEwk6U8xHxSUi-V-VzuzVvJ5a1VyGyCESPP7K3XvRl_8zBNbrbs49kYCbbyOfV2f8gxTx1iDaSvc7AgCGjnYkf1RJt7kAPuo6b1wlLk14dFVgm_MIQBG5fnCuCHBs2h0oxiLQ2N9FbnFVd7wNBUT-m0KoStXkMPY Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Id: None Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Domain-Name: None Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Identity-Status: Confirmed Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Is-Admin-Project: True Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Id: default Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Domain-Name: Default Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Id: 535d117f782341d5aa579a8fb34c35c5 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Project-Name: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Role: reader,member Jul 20 15:30:58.624253 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Roles: reader,member Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2.1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/identity"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.15:8080", "publicURL": "http://10.222.0.15:8080/v1/AUTH_535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/image"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15:9696/"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/compute/v2/535d117f782341d5aa579a8fb34c35c5"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.15/volume/v3/535d117f782341d5aa579a8fb34c35c5"}]}] Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Id: 535d117f782341d5aa579a8fb34c35c5 Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1019241803 Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User: tempest-TestMinimumBasicScenario-1019241803-project Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Id: default Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Domain-Name: Default Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Id: 9b70cbf2ad4f405b84cbea240b83cd05 Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: X-User-Name: tempest-TestMinimumBasicScenario-1019241803-project Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: method: > Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: exception: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. Jul 20 15:30:58.625346 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: action_args: {'id': '95f1b8a6-b4c5-484f-8106-49c1aab6d333'} Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 20 15:30:58.625998 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. Jul 20 15:30:58.626947 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: ERROR nova.api.openstack.wsgi Jul 20 15:30:58.627940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Returning 404 to user: Instance 95f1b8a6-b4c5-484f-8106-49c1aab6d333 could not be found. {{(pid=124396) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 20 15:30:58.627940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-9248482d-994c-493b-8de0-d9f109e999b3 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333" status: 404 len: 111 microversion: 2.1 time: 0.036404 Jul 20 15:30:58.627940 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2141/3068] 10.222.0.15 () {58 vars in 1235 bytes} [Wed Jul 20 15:30:58 2022] GET /compute/v2.1/servers/95f1b8a6-b4c5-484f-8106-49c1aab6d333 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 20 15:30:58.634302 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: DEBUG nova.api.openstack.wsgi [None req-bac0945a-94eb-4038-8639-2b0095ec1e96 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] Calling method '.version_select at 0x7f7b15d06f70>' {{(pid=124396) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 20 15:30:58.653105 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO nova.api.openstack.requestlog [None req-bac0945a-94eb-4038-8639-2b0095ec1e96 tempest-TestMinimumBasicScenario-1019241803 tempest-TestMinimumBasicScenario-1019241803-project] 10.222.0.15 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-653951357" status: 202 len: 0 microversion: 2.1 time: 0.022388 Jul 20 15:30:58.653680 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: [pid: 124396|app: 0|req: 2142/3069] 10.222.0.15 () {58 vars in 1268 bytes} [Wed Jul 20 15:30:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-653951357 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 20 15:33:03.772422 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:03.773109 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:33:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:33:04.543928 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:04.544624 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:33:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:33:08.290325 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:08.291490 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:33:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:33:13.081700 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:13.082530 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:33:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:33:20.963713 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:20.964490 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:33:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:33:28.838950 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:33:28.839864 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:33:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:36:23.782208 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:23.782766 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:36:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:36:24.553033 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:24.553638 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:36:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:36:28.303769 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:28.304450 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:36:28 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!! Jul 20 15:36:33.094374 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:33.094913 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:36:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:36:40.972621 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:40.973211 dv-cinder-849022-4-iscsi devstack@n-api.service[124396]: Wed Jul 20 15:36:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 20 15:36:48.853384 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 20 15:36:48.853983 dv-cinder-849022-4-iscsi devstack@n-api.service[124397]: Wed Jul 20 15:36:48 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6b566c6c-a418-4384-b99e-c7295e754e92 (ip 10.222.0.15) !!!